gristlabs_grist-core/app/plugin
Alex Hall 1864b7ba5d (core) Add BulkAddOrUpdateRecord action for efficiency
Summary:
This diff adds a new `BulkAddOrUpdateRecord` user action which is what is sounds like:

- A bulk version of the existing `AddOrUpdateRecord` action.
- Much more efficient for operating on many records than applying many individual actions.
- Column values are specified as maps from `colId` to arrays of values as usual.
- Produces bulk versions of `AddRecord` and `UpdateRecord` actions instead of many individual actions.

Examples of users wanting to use something like `AddOrUpdateRecord` with large numbers of records:

- https://grist.slack.com/archives/C0234CPPXPA/p1651789710290879
- https://grist.slack.com/archives/C0234CPPXPA/p1660743493480119
- https://grist.slack.com/archives/C0234CPPXPA/p1660333148491559
- https://grist.slack.com/archives/C0234CPPXPA/p1663069291726159

I tested what made many `AddOrUpdateRecord` actions slow in the first place. It was almost entirely due to producing many individual `AddRecord` user actions. About half of that time was for processing the resulting `AddRecord` doc actions. Lookups and updates were not a problem. With these changes, the slowness is gone.

The Python user action implementation is more complex but there are no surprises. The JS API now groups `records` based on the keys of `require` and `fields` so that `BulkAddOrUpdateRecord` can be applied to each group.

Test Plan: Update and extend Python and DocApi tests.

Reviewers: jarek, paulfitz

Reviewed By: jarek, paulfitz

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3642
2022-09-28 17:58:33 +02:00
..
CustomSectionAPI-ti.ts (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling 2022-09-02 16:27:34 +02:00
CustomSectionAPI.ts add a description property to column mapping for custom widgets (#255) 2022-08-26 08:25:34 -04:00
DocApiTypes-ti.ts
DocApiTypes.ts (core) flesh out getAccessToken API documentation 2022-07-28 08:50:12 -04:00
FileParserAPI-ti.ts
FileParserAPI.ts
grist-plugin-api.ts (core) flesh out getAccessToken API documentation 2022-07-28 08:50:12 -04:00
GristAPI-ti.ts (core) add an access token mechanism to help with attachments in custom widgets 2022-07-19 11:55:18 -04:00
GristAPI.ts (core) flesh out getAccessToken API documentation 2022-07-28 08:50:12 -04:00
GristData-ti.ts
GristData.ts (core) flesh out getAccessToken API documentation 2022-07-28 08:50:12 -04:00
GristTable-ti.ts
GristTable.ts
gutil.ts
ImportSourceAPI-ti.ts
ImportSourceAPI.ts
InternalImportSourceAPI-ti.ts
InternalImportSourceAPI.ts
objtypes.ts
PluginManifest-ti.ts
PluginManifest.ts
README.md
RenderOptions-ti.ts
RenderOptions.ts
StorageAPI-ti.ts
StorageAPI.ts
TableOperations.ts
TableOperationsImpl.ts (core) Add BulkAddOrUpdateRecord action for efficiency 2022-09-28 17:58:33 +02:00
tsconfig.json
TypeCheckers.ts
WidgetAPI-ti.ts
WidgetAPI.ts

Methods here are available for use in Grist custom widgets.