Commit Graph

12 Commits

Author SHA1 Message Date
Alex Hall
0de0cb0f4a (core) Add PUT /records DocApi endpoint to AddOrUpdate records
Summary:
As designed in https://grist.quip.com/fZSrAnJKgO5j/Add-or-Update-Records-API

Current `POST /records` adds records, and `PATCH /records` updates them by row ID. This adds `PUT /records` to 'upsert' records, applying the AddOrUpdate user action. PUT was chosen because it's idempotent. Using a separate method (instead of inferring based on the request body) also cleanly separates validation, documentation, etc.

The name `require` for the new property was suggested by Paul because `where` isn't very clear when adding records.

Test Plan: New DocApi tests

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3251
2022-02-12 09:44:34 +02:00
Alex Hall
5d671bf0b3 (core) New type conversion in the backend
Summary: This is https://phab.getgrist.com/D3205 plus some changes (https://github.com/dsagal/grist/compare/type-convert...type-convert-server?expand=1) that move the conversion process to the backend. A new user action ConvertFromColumn uses `call_external` so that the data engine can delegate back to ActiveDoc. Code for creating formatters and parsers is significantly refactored so that most of the logic is in `common` and can be used in different ways.

Test Plan: The original diff adds plenty of tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3240
2022-02-04 20:28:13 +02:00
Alex Hall
8f531ef622 (core) Reference and ReferenceList formatters
Summary:
Previously, ref/reflist columns were formatted entirely based on their visible column, since they received values from the visible or display columns rather than the actual row IDs. This creates `ReferenceFormatter` and `ReferenceListFormatter` which still delegate most of the formatting work to a visible column formatter but fix a few issues:

- ReferenceList columns now actually use the options (e.g. date format) of the visible column to format their elements. Previously they were formatted generically because the visible column formatter wasn't expecting a list.
- Invalid references aren't formatted with an `#Invalid Ref` prefix.
- When the ref column displays the Row ID, it doesn't have a visible or display column. Previously this led to the references being formatted as just numbers in most cases, with special code in the widget to display them like `Table1[2]`. Now they are consistently formatted in that style throughout.

Test Plan: Updated existing tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3212
2022-01-13 18:09:33 +02:00
Jarosław Sadziński
5a876976d5 (core) Preventing empty string update on any column.
Summary: When editor is opened on any column and closed without entering any value, the column is converted to a text column.

Test Plan: browser tests

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3211
2022-01-13 11:05:39 +01:00
Alex Hall
d1a848b44a (core) Parse string cell values in Doc API and Imports
Summary:
- Adds a function `parseUserAction` for parsing strings in UserActions to `ValueParser.ts`
- Adds a boolean option `parseStrings` to use `parseUserAction` in `ActiveDoc.applyUserActions`, off by default.
- Uses `parseStrings` by default in DocApi (set `?noparse=true` in a request to disable) when adding/updating records through the `/data` or `/records` endpoints or in general with the `/apply` endpoint.
- Uses `parseStrings` for various actions in `ActiveDocImport`. Since most types are parsed in Python before these actions are constructed, this only affects references, which still look like errors in the import preview. Importing references can also easily still run into more complicated problems discussed in https://grist.slack.com/archives/C0234CPPXPA/p1639514844028200

Test Plan:
- Added tests to DocApi to compare behaviour with and without string parsing.
- Added a new browser test, fixture doc, and fixture CSV to test importing a file containing references.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3183
2021-12-17 15:40:58 +02:00
Alex Hall
6b448567c9 (core) Refactor more value parsing code into common
Summary:
Following discussion in https://phab.getgrist.com/D3164:

- Change createParser to accept docData and one or two metadata row IDs and let it extract the metadata, so it's more easily usable in the server.
- Change ViewFieldRec.valueParser observable to a function createValueParser.

Test Plan: Existing tests.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3172
2021-12-09 21:05:27 +02:00
Alex Hall
116fb15eda (core) Move most of the reference parsing code into common so that the server can use it
Summary: Refactoring in preparation for parsing strings from the API. The plan is that the API code will only need to do a server-side version of the code in ViewFieldRec.valueParser (minus ReferenceUtils) which is quite minimal.

Test Plan: Nothing extra here, I don't think it's needed. This stuff will get tested more in a future diff which changes the API.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3164
2021-12-07 01:19:18 +02:00
Alex Hall
064455b2f7 (core) Parsing pasted datetimes
Summary:
Add function parseDateTime which parses a string containing both date and time componenents, intended for parsing pasted strings.

Add DateTimeParser subclass of ValueParser.

Test Plan: Extended parseDate.ts and CopyPaste.ts tests.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3152
2021-11-25 00:49:01 +02:00
Alex Hall
0d460ac2d4 (core) Parsing pasted ChoiceLists
Summary:
Added ChoiceListParser capable of parsing JSON, CSVs, and other strings containing user-configured choices (e.g. separated by spaces)

I got a little carried away here. It works, and I can't think of any bugs, but it's complicated enough that there could be hidden edge cases or difficulties maintaining it in the future. The advantage of the current method is that it should work well for ambiguous or poorly formatted inputs, e.g. choices separated only by spaces or choices containing commas which are not escaped/quoted properly. The code can be vastly simplified if we don't try to support that and require that users paste proper JSON or CSVs.

Test Plan: Added a new file test/common/ChoiceListParser.ts with pure unit tests. Waiting for approval of the overall approach before adding to the nbrowser CopyPaste test.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3141
2021-11-20 19:00:36 +02:00
Alex Hall
d63da496a8 (core) Value parsing for refs, parsing data entry for numbers
Summary:
Handle reference columns in ViewFieldRec.valueParser.

Extracted code for reuse from ReferenceEditor to look up values in the visible column. While I was at it, also extracted a bit of common code from ReferenceEditor and ReferenceListEditor into a new class ReferenceUtils. More refactoring could be done in this area but it's out of scope.

Changed NTextEditor to use field.valueParser, which affects numeric and reference fields. In particular this means numbers are parsed on data entry, it doesn't change anything for references.

Test Plan:
Added more CopyPaste testing to test references.

Tested entering slightly formatted numbers in NumberFormatting.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3094
2021-11-01 19:31:52 +02:00
Alex Hall
e58df5df5b (core) ValueParser for Date columns
Summary: Adds parseDateStrict function based on parseDate, uses it in DateParser subclass of ValueParser.

Test Plan:
Tweaked parseDate test to check parseDateStrict.

Extended test in CopyPaste to test parsing dates as well as numbers.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3088
2021-10-26 13:03:19 +02:00
Alex Hall
99878c08ed (core) Add ValueParser, use when pasting
Summary:
Add ValueParser file, base class, and subclasses for column types. Only NumericParser is used for now.

Add valueParser field to ViewFieldRec.

Use valueParser when parsing pasted text data in Grid and Detail views.

Test Plan: Add test to nbrowser CopyPaste suite, copying into a numeric column with different currency and locale settings.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3082
2021-10-21 21:43:19 +02:00