gristlabs_grist-core/app
Alex Hall 1864b7ba5d (core) Add BulkAddOrUpdateRecord action for efficiency
Summary:
This diff adds a new `BulkAddOrUpdateRecord` user action which is what is sounds like:

- A bulk version of the existing `AddOrUpdateRecord` action.
- Much more efficient for operating on many records than applying many individual actions.
- Column values are specified as maps from `colId` to arrays of values as usual.
- Produces bulk versions of `AddRecord` and `UpdateRecord` actions instead of many individual actions.

Examples of users wanting to use something like `AddOrUpdateRecord` with large numbers of records:

- https://grist.slack.com/archives/C0234CPPXPA/p1651789710290879
- https://grist.slack.com/archives/C0234CPPXPA/p1660743493480119
- https://grist.slack.com/archives/C0234CPPXPA/p1660333148491559
- https://grist.slack.com/archives/C0234CPPXPA/p1663069291726159

I tested what made many `AddOrUpdateRecord` actions slow in the first place. It was almost entirely due to producing many individual `AddRecord` user actions. About half of that time was for processing the resulting `AddRecord` doc actions. Lookups and updates were not a problem. With these changes, the slowness is gone.

The Python user action implementation is more complex but there are no surprises. The JS API now groups `records` based on the keys of `require` and `fields` so that `BulkAddOrUpdateRecord` can be applied to each group.

Test Plan: Update and extend Python and DocApi tests.

Reviewers: jarek, paulfitz

Reviewed By: jarek, paulfitz

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3642
2022-09-28 17:58:33 +02:00
..
client (core) Fixing UserManger releted tests 2022-09-27 23:20:53 +02:00
common (core) Add BulkAddOrUpdateRecord action for efficiency 2022-09-28 17:58:33 +02:00
gen-server (core) upgrade typeorm so we can support newer postgres 2022-09-02 15:34:21 -04:00
plugin (core) Add BulkAddOrUpdateRecord action for efficiency 2022-09-28 17:58:33 +02:00
server (core) Add BulkAddOrUpdateRecord action for efficiency 2022-09-28 17:58:33 +02:00
tsconfig.json (core) move home server into core 2020-07-21 20:39:10 -04:00