Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2024-10-27 20:44:07 +00:00
Code Issues Projects Releases Wiki Activity
cf23a2d1ee
gristlabs_grist-core/app/server/lib
History
Paul Fitzpatrick cf23a2d1ee (core) add GVISOR_LIMIT_MEMORY to cap memory available in sandbox 
Summary:
This allows limiting the memory available to documents in the sandbox when gvisor is used. If memory limit is exceeded, we offer to open doc in recovery mode. Recovery mode is tweaked to open docs with tables in "ondemand" mode, which will generally take less memory and allow for deleting rows.

The limit is on the size of the virtual address space available to the sandbox (`RLIMIT_AS`), which in practice appears to function as one would want, and is the only practical option. There is a documented `RLIMIT_RSS` limit to `specifies the limit (in bytes) of the process's resident set (the number of virtual pages resident in RAM)` but this is no longer enforced by the kernel (neither the host nor gvisor).

When the sandbox runs out of memory, there are many ways it can fail. This diff catches all the ones I saw, but there could be more.

Test Plan: added tests

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3398
2022-05-18 14:26:27 -04:00
..
ACLFormula.ts (core) add OWNERS='owners', EDITOR='editors', VIEWER='viewers' to condition formulas 2021-03-19 18:20:33 -04:00
ActionHistory.ts (core) do not look at content of recent actions when loading documents 2021-09-29 11:27:02 -04:00
ActionHistoryImpl.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
ActionSummary.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
ActiveDoc.ts (core) add GVISOR_LIMIT_MEMORY to cap memory available in sandbox 2022-05-18 14:26:27 -04:00
ActiveDocImport.ts (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
AppEndpoint.ts (core) make Grist easier to run with a single server 2022-03-05 13:30:45 -05:00
Authorizer.ts (core) add minimal support for activation keys 2022-05-16 22:46:23 -04:00
BrowserSession.ts (core) Add authSubject and authProvider to sessions 2022-04-11 11:42:02 -07:00
checksumFile.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
Client.ts (core) get user.Name through same mechanism as user.id for websocket Client 2022-04-14 12:49:35 -04:00
Comm.js (core) get user.Name through same mechanism as user.id for websocket Client 2022-04-14 12:49:35 -04:00
dbUtils.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
DiscourseConnect.ts (core) when redirecting, use protocol in APP_HOME_URL if available 2022-04-28 09:13:27 -04:00
DocApi.ts (core) add GVISOR_LIMIT_MEMORY to cap memory available in sandbox 2022-05-18 14:26:27 -04:00
DocClients.ts (core) Add LogMethods helper and use it for more JSON data in logs. Reduce unhelpful logging. 2021-10-25 10:25:18 -04:00
DocManager.ts (core) add GVISOR_LIMIT_MEMORY to cap memory available in sandbox 2022-05-18 14:26:27 -04:00
DocPluginData.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
DocPluginManager.ts (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
DocSession.ts (core) add a user.SessionID value for trigger formulas and granular access rules 2022-02-22 12:50:43 -05:00
DocSnapshots.ts (core) Store monthly snapshots for 8 years to give Enterprise plans a more significant advantage 2022-04-05 18:11:13 +02:00
DocStorage.ts (core) Add attachment and data size usage 2022-05-04 13:46:55 -07:00
DocStorageManager.ts (core) Add API endpoint to get site usage summary 2022-05-16 11:16:19 -07:00
docUtils.d.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
docUtils.js Correct spelling mistakes 2022-02-19 09:46:49 +00:00
DocWorker.ts (core) Clean up and refactor uses of HomeDBManager.getDoc 2022-03-24 13:42:36 +02:00
DocWorkerMap.ts (core) Add flexibility to daily API usage limit 2022-04-28 16:22:18 +02:00
ExcelFormatter.ts (core) Custom Widget column mapping feature. 2022-02-08 17:41:04 +01:00
ExpandedQuery.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
Export.ts (core) Use MetaTableData more 2021-12-07 17:09:58 +02:00
ExportCSV.ts (core) Use MetaTableData more 2021-12-07 17:09:58 +02:00
ExportXLSX.ts (core) Exposing more descriptive errors from exports 2021-11-30 17:26:32 +01:00
expressWrap.ts (core) Include altSessionId in logs 2022-04-08 16:40:34 -07:00
ExternalStorage.ts (core) add machinery for self-managed flavor of Grist 2022-05-12 12:39:52 -04:00
extractOrg.ts (core) when redirecting, use protocol in APP_HOME_URL if available 2022-04-28 09:13:27 -04:00
FileParserElement.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
filterUtils.ts (core) uncheck FullCopy special when copying/forking a document 2021-04-29 08:56:54 -04:00
FlexServer.ts (core) Move Notifier to /ext 2022-05-18 08:02:32 -07:00
ForwardAuthLogin.ts (core) add a login method based on headers 2022-04-04 18:36:09 -04:00
GoogleAuth.ts (core) Exposing more descriptive errors from exports 2021-11-30 17:26:32 +01:00
GoogleExport.ts (core) Include altSessionId in logs 2022-04-08 16:40:34 -07:00
GoogleImport.ts (core) Extending Google Drive integration scope 2021-10-01 10:47:12 +02:00
GranularAccess.ts (core) add GVISOR_LIMIT_MEMORY to cap memory available in sandbox 2022-05-18 14:26:27 -04:00
GristServer.ts (core) Move Notifier to /ext 2022-05-18 08:02:32 -07:00
gristSessions.ts (core) Implement DiscourseConnect to enable easy sign-in to community forum 2021-10-01 11:24:22 -04:00
guessExt.ts (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
HashUtil.ts (core) add more detail to /compare endpoint 2020-09-18 16:31:29 -04:00
HostedMetadataManager.ts (core) Add API endpoint to get site usage summary 2022-05-16 11:16:19 -07:00
HostedStorageManager.ts (core) Add API endpoint to get site usage summary 2022-05-16 11:16:19 -07:00
IBilling.ts (core) add minimal support for activation keys 2022-05-16 22:46:23 -04:00
IChecksumStore.ts (core) revamp snapshot inventory 2020-10-30 13:52:46 -04:00
ICreate.ts (core) Move Notifier to /ext 2022-05-18 08:02:32 -07:00
IDocStorageManager.ts (core) Add API endpoint to get site usage summary 2022-05-16 11:16:19 -07:00
idUtils.ts (core) Move Notifier to /ext 2022-05-18 08:02:32 -07:00
IElectionStore.ts (core) move some material to core that slipped through in a rebase 2020-07-23 11:29:05 -04:00
initialDocSql.ts (core) Distinct style rules for summary columns 2022-04-27 20:51:23 +02:00
INotifier.ts (core) add a tool for deleting a user 2021-09-29 12:08:23 -04:00
ISandbox.ts (core) New type conversion in the backend 2022-02-04 20:28:13 +02:00
IShell.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
ITestingHooks-ti.ts (core) be careful when reassigning a doc to a worker it was on before 2022-03-08 17:20:01 -05:00
ITestingHooks.ts (core) be careful when reassigning a doc to a worker it was on before 2022-03-08 17:20:01 -05:00
log.ts (core) give instructions on using Grist with docker 2020-10-28 13:59:13 -04:00
LogMethods.ts (core) Add LogMethods helper and use it for more JSON data in logs. Reduce unhelpful logging. 2021-10-25 10:25:18 -04:00
manifest.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
MinimalLogin.ts (core) Add authSubject and authProvider to sessions 2022-04-11 11:42:02 -07:00
NSandbox.ts (core) add GVISOR_LIMIT_MEMORY to cap memory available in sandbox 2022-05-18 14:26:27 -04:00
OnDemandActions.ts (core) add GVISOR_LIMIT_MEMORY to cap memory available in sandbox 2022-05-18 14:26:27 -04:00
PermissionInfo.ts (core) Add 'user' variable to trigger formulas 2021-07-15 15:18:32 -07:00
Permit.ts (core) Add new Grist login page 2022-04-01 15:24:19 -07:00
places.ts (core) fix docker packaging after core shuffle 2020-07-22 14:45:42 -04:00
PluginEndpoint.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
PluginManager.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
reportTimeTaken.ts (core) Log the time taken by decodeActionFromRow() operations. 2021-08-20 11:28:33 -04:00
requestUtils.ts (core) Add API endpoint to get site usage summary 2022-05-16 11:16:19 -07:00
RowAccess.ts (core) Add rules to eslint to better match our coding conventions. 2021-05-24 12:56:18 -04:00
SafePythonComponent.ts (core) Move file import plugins into core/sandbox/grist 2021-08-09 18:37:14 +02:00
SamlConfig.ts (core) when redirecting, use protocol in APP_HOME_URL if available 2022-04-28 09:13:27 -04:00
SandboxControl.ts (core) freshen tests for python3 2021-11-10 10:46:12 -05:00
sandboxUtil.js (core) move home server into core 2020-07-21 20:39:10 -04:00
sendAppPage.ts (core) add minimal support for activation keys 2022-05-16 22:46:23 -04:00
ServerColumnGetters.ts (core) Adding sort options for columns. 2021-11-03 15:31:39 +01:00
ServerLocale.ts (core) Simple localization support and currency selector. 2021-08-26 13:36:49 -07:00
serverUtils.ts (core) support setting python version of new docs with PYTHON_VERSION_ON_CREATION 2021-11-05 10:51:18 -04:00
Sessions.ts (core) make Grist easier to run with a single server 2022-03-05 13:30:45 -05:00
Sharing.ts (core) Add API endpoint to get site usage summary 2022-05-16 11:16:19 -07:00
shortDesc.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
shutdown.js (core) move home server into core 2020-07-21 20:39:10 -04:00
SQLiteDB.ts (core) Remove expired attachments every hour and on shutdown 2022-04-22 20:43:59 +02:00
TableMetadataLoader.ts (core) open documents without blocking on data engine 2021-10-01 10:18:56 -04:00
TagChecker.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
TestingHooks.ts (core) be careful when reassigning a doc to a worker it was on before 2022-03-08 17:20:01 -05:00
TestLogin.ts (core) Add new Grist login page 2022-04-01 15:24:19 -07:00
Throttle.ts (core) tweak throttling to work for gvisor/runsc 2021-11-04 17:23:43 -04:00
TimeQuery.ts (core) Raw renames 2022-04-27 22:21:55 +02:00
Triggers.ts (core) Switch webhook secret cache from LRU to TTL so that unsubscribing can drain the queue 2022-05-16 18:02:19 +02:00
UnsafeNodeComponent.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
uploads.ts (core) Include altSessionId in logs 2022-04-08 16:40:34 -07:00
WidgetRepository.ts (core) Widget options api 2022-01-13 11:10:17 +01:00
WorkCoordinator.ts (core) move home server into core 2020-07-21 20:39:10 -04:00