gristlabs_grist-core/test
Alex Hall 1864b7ba5d (core) Add BulkAddOrUpdateRecord action for efficiency
Summary:
This diff adds a new `BulkAddOrUpdateRecord` user action which is what is sounds like:

- A bulk version of the existing `AddOrUpdateRecord` action.
- Much more efficient for operating on many records than applying many individual actions.
- Column values are specified as maps from `colId` to arrays of values as usual.
- Produces bulk versions of `AddRecord` and `UpdateRecord` actions instead of many individual actions.

Examples of users wanting to use something like `AddOrUpdateRecord` with large numbers of records:

- https://grist.slack.com/archives/C0234CPPXPA/p1651789710290879
- https://grist.slack.com/archives/C0234CPPXPA/p1660743493480119
- https://grist.slack.com/archives/C0234CPPXPA/p1660333148491559
- https://grist.slack.com/archives/C0234CPPXPA/p1663069291726159

I tested what made many `AddOrUpdateRecord` actions slow in the first place. It was almost entirely due to producing many individual `AddRecord` user actions. About half of that time was for processing the resulting `AddRecord` doc actions. Lookups and updates were not a problem. With these changes, the slowness is gone.

The Python user action implementation is more complex but there are no surprises. The JS API now groups `records` based on the keys of `require` and `fields` so that `BulkAddOrUpdateRecord` can be applied to each group.

Test Plan: Update and extend Python and DocApi tests.

Reviewers: jarek, paulfitz

Reviewed By: jarek, paulfitz

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3642
2022-09-28 17:58:33 +02:00
..
client (core) Fixing ACIndex highlightMatches functions 2022-08-26 08:34:16 +02:00
common give parseDate test more time (#278) 2022-09-09 17:05:29 -04:00
fixtures (core) Moving widget tests to core 2022-09-06 17:17:14 +02:00
gen-server (core) upgrade typeorm so we can support newer postgres 2022-09-02 15:34:21 -04:00
nbrowser (core) tweak duplicate document test to be more reliable 2022-09-22 14:26:56 -04:00
server (core) Add BulkAddOrUpdateRecord action for efficiency 2022-09-28 17:58:33 +02:00
chai-as-promised.js (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
init-mocha-webdriver.js Correct spelling mistakes 2022-02-19 09:46:49 +00:00
mocha.opts (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
report-why-tests-hang.js (core) Move report-why-tests-hang helper to core 2021-04-26 23:52:16 -04:00
test_under_docker.sh Skipping tests that can't be run on external server. 2022-08-12 17:23:22 +02:00
testUtils.ts (core) add missing tsconfig file that affects IDEs 2022-05-27 13:48:58 -04:00
tsconfig.json (core) Faster builds all around. 2022-07-04 10:42:40 -04:00
utils.js (core) Moving client and common tests to core 2022-08-23 19:20:10 +02:00
xunit-file.js (core) Parallelize jenkins by running on multiple machines. 2022-07-01 16:18:36 -04:00