gristlabs_grist-core/test/server
Alex Hall 1864b7ba5d (core) Add BulkAddOrUpdateRecord action for efficiency
Summary:
This diff adds a new `BulkAddOrUpdateRecord` user action which is what is sounds like:

- A bulk version of the existing `AddOrUpdateRecord` action.
- Much more efficient for operating on many records than applying many individual actions.
- Column values are specified as maps from `colId` to arrays of values as usual.
- Produces bulk versions of `AddRecord` and `UpdateRecord` actions instead of many individual actions.

Examples of users wanting to use something like `AddOrUpdateRecord` with large numbers of records:

- https://grist.slack.com/archives/C0234CPPXPA/p1651789710290879
- https://grist.slack.com/archives/C0234CPPXPA/p1660743493480119
- https://grist.slack.com/archives/C0234CPPXPA/p1660333148491559
- https://grist.slack.com/archives/C0234CPPXPA/p1663069291726159

I tested what made many `AddOrUpdateRecord` actions slow in the first place. It was almost entirely due to producing many individual `AddRecord` user actions. About half of that time was for processing the resulting `AddRecord` doc actions. Lookups and updates were not a problem. With these changes, the slowness is gone.

The Python user action implementation is more complex but there are no surprises. The JS API now groups `records` based on the keys of `require` and `fields` so that `BulkAddOrUpdateRecord` can be applied to each group.

Test Plan: Update and extend Python and DocApi tests.

Reviewers: jarek, paulfitz

Reviewed By: jarek, paulfitz

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3642
2022-09-28 17:58:33 +02:00
..
lib (core) Add BulkAddOrUpdateRecord action for efficiency 2022-09-28 17:58:33 +02:00
Comm.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
customUtil.ts (core) Fix nbrowser/CopyPaste test on Mac 2022-07-29 14:08:29 +02:00
docTools.ts (core) add a yarn run cli tool, and add a sqlite gristify option 2022-07-14 12:00:30 -04:00
gristClient.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
testUtils.ts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00