gristlabs_grist-core/app/server
Alex Hall 1864b7ba5d (core) Add BulkAddOrUpdateRecord action for efficiency
Summary:
This diff adds a new `BulkAddOrUpdateRecord` user action which is what is sounds like:

- A bulk version of the existing `AddOrUpdateRecord` action.
- Much more efficient for operating on many records than applying many individual actions.
- Column values are specified as maps from `colId` to arrays of values as usual.
- Produces bulk versions of `AddRecord` and `UpdateRecord` actions instead of many individual actions.

Examples of users wanting to use something like `AddOrUpdateRecord` with large numbers of records:

- https://grist.slack.com/archives/C0234CPPXPA/p1651789710290879
- https://grist.slack.com/archives/C0234CPPXPA/p1660743493480119
- https://grist.slack.com/archives/C0234CPPXPA/p1660333148491559
- https://grist.slack.com/archives/C0234CPPXPA/p1663069291726159

I tested what made many `AddOrUpdateRecord` actions slow in the first place. It was almost entirely due to producing many individual `AddRecord` user actions. About half of that time was for processing the resulting `AddRecord` doc actions. Lookups and updates were not a problem. With these changes, the slowness is gone.

The Python user action implementation is more complex but there are no surprises. The JS API now groups `records` based on the keys of `require` and `fields` so that `BulkAddOrUpdateRecord` can be applied to each group.

Test Plan: Update and extend Python and DocApi tests.

Reviewers: jarek, paulfitz

Reviewed By: jarek, paulfitz

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3642
2022-09-28 17:58:33 +02:00
..
declarations (core) disentangle some server tests, release to core, add GRIST_PROXY_AUTH_HEADER test 2022-03-24 15:11:32 -04:00
lib (core) Add BulkAddOrUpdateRecord action for efficiency 2022-09-28 17:58:33 +02:00
utils (core) add a yarn run cli tool, and add a sqlite gristify option 2022-07-14 12:00:30 -04:00
companion.ts (core) upgrade typeorm so we can support newer postgres 2022-09-02 15:34:21 -04:00
declarations.d.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
devServerMain.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
generateCheckpoint.ts (core) get all tests working under python3/gvisor 2021-10-18 17:44:15 -04:00
generateInitialDocSql.ts (core) get all tests working under python3/gvisor 2021-10-18 17:44:15 -04:00
mergedServerMain.ts (core) dust off electron build a little bit 2022-07-29 11:19:26 -04:00
tsconfig.json (core) Simple localization support and currency selector. 2021-08-26 13:36:49 -07:00