Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2026-03-02 04:09:24 +00:00
Code Issues Projects Releases Wiki Activity
Files
efb5fc63d73af2d48c1aff5f04d918741285161a
gristlabs_grist-core/app/server/lib
History
Dmitry S 526a5df157 (core) Manage memory used for websocket responses to reduce the risk of server crashes. 
Summary:
- Implements MemoryPool for waiting on memory reservations.
- Uses MemoryPool to control memory used for stringifying JSON responses in Client.ts
- Limits total size of _missedMessages that may be queued for a particular client.
- Upgrades ws library, which may reduce memory usage, and allows pausing the websocket for testing.
  - The upgrade changed subtle behavior corners, requiring various fixes to code and tests.

- dos.ts:
  - Includes Paul's fixes and updates to the dos.ts script for manual stress-testing.
  - Logging tweaks, to avoid excessive dumps on uncaughtError, and include timestamps.

Test Plan:
- Includes a test that measures heap size, and fails without memory management.
- Includes a unittest for MemoryPool
- Some cleanup and additions to TestServer helper; in particular adds makeUserApi() helper used in multiple tests.
- Some fixes related to ws upgrade.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3974
2023-08-07 11:28:31 -04:00
..
AccessTokens.ts
(core) add an access token mechanism to help with attachments in custom widgets
2022-07-19 11:55:18 -04:00
ACLFormula.ts
(core) add OWNERS='owners', EDITOR='editors', VIEWER='viewers' to condition formulas
2021-03-19 18:20:33 -04:00
ActionHistory.ts
(core) Fix issue with lodash's map interpreting objects with length as array-like
2023-07-01 21:07:39 -04:00
ActionHistoryImpl.ts
support other SQLite wrappers, and various hooks needed by grist-static (#516)
2023-05-23 15:17:28 -04:00
ActiveDoc.ts
move getTemplateOrg method; enable template org in docker tests (#602)
2023-07-31 16:10:59 -04:00
ActiveDocImport.ts
Split out new importFileAsNewTable method for grist-static (#564)
2023-07-12 15:57:02 +02:00
AppEndpoint.ts
move getTemplateOrg method; enable template org in docker tests (#602)
2023-07-31 16:10:59 -04:00
AppSettings.ts
(core) give preliminary support in core for storing snapshots in S3-compatible stores via minio-js client
2022-12-21 11:41:31 -05:00
Assistance.ts
(core) Show tweaked formula in AI responses
2023-08-04 01:31:31 -07:00
Authorizer.ts
(core) Add optional telemetry to grist-core
2023-06-07 12:00:51 -04:00
BrowserSession.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
checksumFile.ts
(core) move home server into core
2020-07-21 20:39:10 -04:00
Client.ts
(core) Manage memory used for websocket responses to reduce the risk of server crashes.
2023-08-07 11:28:31 -04:00
Comm.ts
(core) Manage memory used for websocket responses to reduce the risk of server crashes.
2023-08-07 11:28:31 -04:00
configureMinIOExternalStorage.ts
Abort when MinIO bucket does not have versioning enabled #545 (#546)
2023-07-10 06:24:55 -04:00
dbUtils.ts
(core) remove ormconfig.js from saas build; more sendgrid logging
2023-03-01 17:02:47 -05:00
DiscourseConnect.ts
(core) Adding GristConnect login system
2022-05-18 20:28:25 +02:00
DocApi.ts
(core) deleting queue from single webhook
2023-07-18 11:46:10 +02:00
DocClients.ts
(core) add a yarn run cli tool, and add a sqlite gristify option
2022-07-14 12:00:30 -04:00
DocManager.ts
(core) port some more test/browser tests to newer selenium
2023-04-12 13:00:53 -04:00
DocPluginData.ts
(core) move home server into core
2020-07-21 20:39:10 -04:00
DocPluginManager.ts
(core) add an access token mechanism to help with attachments in custom widgets
2022-07-19 11:55:18 -04:00
DocSession.ts
(core) User language switcher
2023-01-26 09:47:14 +01:00
DocSnapshots.ts
Track saved version per hour, day, week, month, year, and number of times a version with parameter (#509)
2023-05-12 11:38:29 -04:00
DocStorage.ts
fix a node-sqlite3-ism that breaks record removal in grist-static (#566)
2023-07-11 05:52:06 -04:00
DocStorageManager.ts
(core) dust off electron build a little bit
2022-07-29 11:19:26 -04:00
docUtils.d.ts
(core) move home server into core
2020-07-21 20:39:10 -04:00
docUtils.js
Correct spelling mistakes
2022-02-19 09:46:49 +00:00
DocWorker.ts
(core) Billing for formula assistant
2023-07-10 13:24:08 +02:00
DocWorkerMap.ts
(core) Add unquarantine command to admin CLI
2022-08-15 13:04:55 -07:00
ExcelFormatter.ts
(core) Currency from grist column is persistent when exporting to excel
2023-05-08 10:39:01 +02:00
ExpandedQuery.ts
Correct spelling mistakes
2022-02-19 09:46:49 +00:00
Export.ts
(core) Filter rows based on linked widgets when exporting view
2023-07-26 21:49:52 +02:00
ExportCSV.ts
(core) Filter rows based on linked widgets when exporting view
2023-07-26 21:49:52 +02:00
ExportTableSchema.ts
(core) Fix imports into reference columns, and support two ways to import Numeric as a reference.
2023-05-02 10:28:14 -04:00
ExportXLSX.ts
(core) Filter rows based on linked widgets when exporting view
2023-07-26 21:49:52 +02:00
expressWrap.ts
(core) For exporting XLSX, do it memory-efficiently in a worker thread.
2023-06-01 12:06:48 -04:00
ExternalStorage.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
extractOrg.ts
(core) upgrade typeorm so we can support newer postgres
2022-09-02 15:34:21 -04:00
FileParserElement.ts
(core) move home server into core
2020-07-21 20:39:10 -04:00
filterUtils.ts
(core) uncheck FullCopy special when copying/forking a document
2021-04-29 08:56:54 -04:00
FlexServer.ts
External storage: split checkBackend and configure (follow-up #545) (#567)
2023-07-13 02:44:46 -04:00
ForwardAuthLogin.ts
(core) clean up interaction of forward auth with session
2022-06-15 13:06:12 -04:00
GoogleAuth.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
GoogleExport.ts
(core) For exporting XLSX, do it memory-efficiently in a worker thread.
2023-06-01 12:06:48 -04:00
GoogleImport.ts
(core) Adding DELETE /api/docs/webhooks/queue endpoint to clear the queue
2022-12-01 12:23:19 +01:00
GranularAccess.ts
support other SQLite wrappers, and various hooks needed by grist-static (#516)
2023-05-23 15:17:28 -04:00
GristServer.ts
(core) Add Support Grist page and nudge
2023-07-04 17:36:59 -04:00
gristSessions.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
gristSettings.ts
move getTemplateOrg method; enable template org in docker tests (#602)
2023-07-31 16:10:59 -04:00
guessExt.ts
(core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows
2022-05-12 14:43:21 +02:00
hashingUtils.ts
(core) Add Support Grist page and nudge
2023-07-04 17:36:59 -04:00
HashUtil.ts
(core) add more detail to /compare endpoint
2020-09-18 16:31:29 -04:00
HostedMetadataManager.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
HostedStorageManager.ts
(core) Persist forks in home db
2023-02-20 22:46:36 -05:00
httpEncoding.ts
(core) External requests
2022-06-17 21:53:20 +02:00
IBilling.ts
(core) Add basic activation page to grist-ee
2022-08-23 10:30:52 -07:00
IChecksumStore.ts
(core) revamp snapshot inventory
2020-10-30 13:52:46 -04:00
ICreate.ts
External storage: split checkBackend and configure (follow-up #545) (#567)
2023-07-13 02:44:46 -04:00
IDocStorageManager.ts
(core) add a yarn run cli tool, and add a sqlite gristify option
2022-07-14 12:00:30 -04:00
idUtils.ts
(core) Persist forks in home db
2023-02-20 22:46:36 -05:00
IElectionStore.ts
(core) move some material to core that slipped through in a rebase
2020-07-23 11:29:05 -04:00
initialDocSql.ts
(core) reconcile webhook and widget description migrations
2023-05-15 11:56:15 -04:00
INotifier.ts
(core) Speed up and upgrade build.
2022-06-27 16:10:10 -04:00
ISandbox.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
IShell.ts
(core) dust off electron build a little bit
2022-07-29 11:19:26 -04:00
ITestingHooks-ti.ts
(core) Manage memory used for websocket responses to reduce the risk of server crashes.
2023-08-07 11:28:31 -04:00
ITestingHooks.ts
(core) Manage memory used for websocket responses to reduce the risk of server crashes.
2023-08-07 11:28:31 -04:00
log.ts
fix log.add under electron (#478)
2023-04-03 13:29:35 -04:00
LogMethods.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
manifest.ts
(core) move home server into core
2020-07-21 20:39:10 -04:00
MemoryPool.ts
(core) Manage memory used for websocket responses to reduce the risk of server crashes.
2023-08-07 11:28:31 -04:00
MinimalLogin.ts
A set of tweaks to simplify electron packaging (#421)
2023-02-13 15:52:17 -05:00
MinIOExternalStorage.ts
Issue 359 support scaleway (#577)
2023-07-20 06:16:58 -04:00
NSandbox.ts
fix type issue if python is missing when running unsandboxed (#610)
2023-08-05 11:24:01 -04:00
NullSandbox.ts
(core) Fix snapshot migrations
2023-05-23 15:50:15 -04:00
OnDemandActions.ts
(core) Adds a UI panel for managing webhooks
2023-05-08 18:25:27 -04:00
PermissionInfo.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
Permit.ts
(core) Add new Grist login page
2022-04-01 15:24:19 -07:00
places.ts
A set of tweaks to simplify electron packaging (#421)
2023-02-13 15:52:17 -05:00
PluginEndpoint.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
PluginManager.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
ProcessMonitor.ts
(core) Add optional telemetry to grist-core
2023-06-07 12:00:51 -04:00
ProxyAgent.ts
(core) Add logging of errors whenever ProxyAgent is used, and a test for it.
2023-05-17 10:21:53 -04:00
reportTimeTaken.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
Requests.ts
REQUEST now supports POST (#588)
2023-07-30 15:13:43 -04:00
requestUtils.ts
(core) Add Support Grist page and nudge
2023-07-04 17:36:59 -04:00
RowAccess.ts
(core) control the distribution of attachment metadata
2022-12-22 09:10:30 -05:00
SafePythonComponent.ts
(core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling
2022-09-02 16:27:34 +02:00
SamlConfig.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
SandboxControl.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
sandboxUtil.ts
(core) Update logging in sandbox code, and log tracebacks as single log messages.
2023-07-18 11:21:25 -04:00
sendAppPage.ts
move getTemplateOrg method; enable template org in docker tests (#602)
2023-07-31 16:10:59 -04:00
ServerColumnGetters.ts
(core) Filter rows based on linked widgets when exporting view
2023-07-26 21:49:52 +02:00
ServerLocale.ts
(core) Converting server-side Comm.js to typescript
2022-06-07 15:47:17 -04:00
serverUtils.ts
(core) Add optional telemetry to grist-core
2023-06-07 12:00:51 -04:00
Sessions.ts
(core) Converting server-side Comm.js to typescript
2022-06-07 15:47:17 -04:00
Sharing.ts
(core) port some more test/browser tests to newer selenium
2023-04-12 13:00:53 -04:00
shortDesc.ts
(core) move home server into core
2020-07-21 20:39:10 -04:00
shutdown.js
support other SQLite wrappers, and various hooks needed by grist-static (#516)
2023-05-23 15:17:28 -04:00
SqliteCommon.ts
support other SQLite wrappers, and various hooks needed by grist-static (#516)
2023-05-23 15:17:28 -04:00
SQLiteDB.ts
support other SQLite wrappers, and various hooks needed by grist-static (#516)
2023-05-23 15:17:28 -04:00
SqliteNode.ts
support other SQLite wrappers, and various hooks needed by grist-static (#516)
2023-05-23 15:17:28 -04:00
TableMetadataLoader.ts
(core) open documents without blocking on data engine
2021-10-01 10:18:56 -04:00
TagChecker.ts
(core) move home server into core
2020-07-21 20:39:10 -04:00
Telemetry.ts
(core) Add telemetry for AI Assistant
2023-07-20 12:50:26 -04:00
TestingHooks.ts
(core) Manage memory used for websocket responses to reduce the risk of server crashes.
2023-08-07 11:28:31 -04:00
TestLogin.ts
(core) make the support account configurable, and allow listing public sites
2022-10-14 14:23:59 -04:00
Throttle.ts
(core) Fix bugs with intervals
2022-08-25 12:38:36 -07:00
TimeQuery.ts
(core) Adds a UI panel for managing webhooks
2023-05-08 18:25:27 -04:00
Triggers.ts
(core) GET endpoint for webhooks returns now data in format {webhooks:[...]}
2023-07-26 11:36:24 +02:00
UnsafeNodeComponent.ts
(core) Faster builds all around.
2022-07-04 10:42:40 -04:00
uploads.ts
(core) remove deprecated /download endpoint in favor of newer /api/docs/NNNN/download
2022-09-20 15:26:04 -04:00
WidgetRepository.ts
(core) A few assorted one-line fixes
2023-01-09 23:31:04 -05:00
WorkCoordinator.ts
support other SQLite wrappers, and various hooks needed by grist-static (#516)
2023-05-23 15:17:28 -04:00
workerExporter.ts
(core) Filter rows based on linked widgets when exporting view
2023-07-26 21:49:52 +02:00