Commit Graph

12 Commits

Author SHA1 Message Date
Alex Hall
9fffb491f9 (core) External requests
Summary:
Adds a Python function `REQUEST` which makes an HTTP GET request. Behind the scenes it:

- Raises a special exception to stop trying to evaluate the current cell and just keep the existing value.
- Notes the request arguments which will be returned by `apply_user_actions`.
- Makes the actual request in NodeJS, which sends back the raw response data in a new action `RespondToRequests` which reevaluates the cell(s) that made the request.
- Wraps the response data in a class which mimics the `Response` class of the `requests` library.

In certain cases, this asynchronous flow doesn't work and the sandbox will instead synchronously call an exported JS method:

- When reevaluating a single cell to get a formula error, the request is made synchronously.
- When a formula makes multiple requests, the earlier responses are retrieved synchronously from files which store responses as long as needed to complete evaluating formulas. See https://grist.slack.com/archives/CL1LQ8AT0/p1653399747810139

Test Plan: Added Python and nbrowser tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: paulfitz, dsagal

Differential Revision: https://phab.getgrist.com/D3429
2022-06-17 21:53:20 +02:00
Alex Hall
c5ebd7db3d (core) Add PEEK() function to bypass circular dependencies
Summary:
Adds a Python function `PEEK()` for use in formulas which temporarily sets a new attribute `Engine._peeking` which disables the `_use_node` method, preventing dependency tracking and allowing the given expression to use outdated values. This allows circumventing circular reference errors. It's particularly meant for trigger formulas although it works in normal formulas as well. The expression is wrapped in a `lambda` by `codebuilder` for lazy evaluation.

Discussion: https://grist.slack.com/archives/C0234CPPXPA/p1653571024031359

Test Plan: Added a Python unit test for circular trigger formulas using PEEK.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3453
2022-06-02 19:50:14 +02:00
Alex Hall
2d0978559b (core) Replace compute stack and frames with _current_node and _current_row_id
Summary:
This is an attempt to optimise Engine._use_node. It doesn't seem to actually improve overall performance significantly, but it shouldn't make it worse, and I think it's an improvement to the code.

It turns out that there's no need to track a stack of compute frames any more. The only time we get close to nested evaluation, we set allow_evaluation=False to prevent it actually happening. So there's only one 'frame' during actual evaluation, which means we can get rid of the concept of frames entirely. This allows simplifying the code and letting the computer do less work in general.

Test Plan: this

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3310
2022-03-11 12:34:00 +02:00
Dmitry S
3136077636 (core) Update documentation of CURRENT_CONVERSION function
Summary: As requested by Anais in https://github.com/dsagal/grist-help/pull/132

Test Plan: Only a change to doc-comment (from which help docs are built).

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3267
2022-02-16 08:48:12 -05:00
Alex Hall
5d671bf0b3 (core) New type conversion in the backend
Summary: This is https://phab.getgrist.com/D3205 plus some changes (https://github.com/dsagal/grist/compare/type-convert...type-convert-server?expand=1) that move the conversion process to the backend. A new user action ConvertFromColumn uses `call_external` so that the data engine can delegate back to ActiveDoc. Code for creating formatters and parsers is significantly refactored so that most of the logic is in `common` and can be used in different ways.

Test Plan: The original diff adds plenty of tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3240
2022-02-04 20:28:13 +02:00
Alex Hall
0e5c2bee59 (core) Add CellError to explain when a formula refers to a cell with an error value
Summary:
get_cell_value wraps RaisedException with CellError to expand the error message for the user.

This is still pretty conceptual, the comments explain some things to think about, but it works and is an improvement.

Test Plan: Updated Python unit tests

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2928
2021-07-30 22:58:45 +02:00
Alex Hall
04e5d90f86 (core) Barely working reference lists in frontend
Summary:
This makes it possible to set the type of a column to ReferenceList, but the UI is terrible

ReferenceList.ts is a mishmash of ChoiceList and Reference that sort of works but something about the CSS is clearly broken

ReferenceListEditor is just a text editor, you have to type in a JSON array of row IDs. Ignore the value that's present when you start editing. I can maybe try mashing together ReferenceEditor and ChoiceListEditor but it doesn't seem wise.
I think @georgegevoian should take over here. Reviewing the diff as it is to check for obvious issues is probably good but I don't think it's worth trying to land/merge anything.

Test Plan: none

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: georgegevoian

Differential Revision: https://phab.getgrist.com/D2914
2021-07-23 18:41:44 +02:00
Alex Hall
16f297a250 (core) Simple Python 3 compatibility changes
Summary: Changes that move towards python 3 compatibility that are easy to review without much thought

Test Plan: The tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2873
2021-06-22 17:13:17 +02:00
Dmitry S
003029bf8a (core) Improve suggestions for formula autocomplete
Summary:
- Make suggestions less case-sensitive (not entirely case-insensitive, but
  allow top-level suggestions to match in all-lowercase)
- Add function signatures to suggestions for Grist functions.
- Excel-like functions that are present but not implemented are no longer
  offered as suggestions.

Test Plan:
Added a test case on python side, and a browser test case for how suggestions
are rendered and inserted.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2608
2020-09-12 00:21:29 -04:00
Dmitry S
7a8debae16 (core) Improve object serialization, to help get RECORD data to Custom Widgets.
Summary:
- Change RECORD's dates_as_str default to False.
- Reimplement objtype encode_object/decode_object with less machinery.
- Implement encoding of dicts (with string keys).
- Make lists and dicts encode values recursively.
- Implement encoding/decoding in the client
- Decode automatically in plugins' fetchSelectedTable/Record, with an option to skip.

Test Plan: Tested manually, not sure what tests may be affected yet.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2593
2020-08-21 18:33:28 -04:00
Dmitry S
4e11b6a922 (core) Implement RECORD function that converts Records to Python dictionaries
Summary:
Examining a Record is currently difficult, because its columns are hard to
list (and to use), and CircularRef errors hard to avoid. The RECORD function
takes care of this mess to return a simple dictionary of values.

- Supports dates_as_iso=False flag to turn off the translation of date/datetime
  objects to strings.
- Supports expand_refs=True flag to apply RECORD() to encountered values of
  type Record, for a single level of nesting.

Test Plan: Added a unittest for RECORD()

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2582
2020-08-12 13:31:57 -04:00
Paul Fitzpatrick
b82eec714a (core) move data engine code to core
Summary:
this moves sandbox/grist to core, and adds a requirements.txt
file for reconstructing the content of sandbox/thirdparty.

Test Plan:
existing tests pass.
Tested core functionality manually.  Tested docker build manually.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2563
2020-07-29 08:57:25 -04:00