Commit Graph

15 Commits

Author SHA1 Message Date
Alex Hall
b9adcefcce (core) Use new asttokens.ASTText to support dollar signs inside f-strings
Summary:
Replaced uses of asttokens.ASTTokens with asttokens.ASTText when working with plain `ast` trees, and use `atok.get_text_range` instead of `node.first_token`.

Upgraded asttokens in Python 2 (it was already upgraded in Python 3).

Test Plan: Added a test with f-strings.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D4001
2023-08-23 13:39:28 +02:00
Dmitry S
534615dd50 (core) Update logging in sandbox code, and log tracebacks as single log messages.
Summary:
- Replace logger module by the standard module 'logging'.
- When a log message from the sandbox includes newlines (e.g. for tracebacks),
  keep those lines together in the Node log message.

  Previously each line was a different message, making it difficult to view
  tracebacks, particularly in prod where each line becomes a separate message
  object.

- Fix assorted lint errors.

Test Plan: Added a test for the log-line splitting and escaping logic.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3956
2023-07-18 11:21:25 -04:00
Dmitry S
b4cc519616 (core) Ignore leading whitespace in formulas, and strip out leading '=' sign users might add
Summary:
This addresses two issues, differently:
- For a formula with leading whitespace, like " 1+1", it is stored as is, but
  is fixed to work (it should be valid Python, and whitespace is only stripped out
  at parsing time to avoid intentation errors caused by the way it gets parsed)
- For a formula with a leading equals-sign ("="), it is stripped out on the
  client side before the formula is stored. Grist documentation uses leading
  "=" to indicate formulas (because UI shows an "=" icon), and Excel formulas
  actually contain the leading "=", so it is a common mistake to include it.

Test Plan: Added new test cases

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3873
2023-04-25 15:28:40 -04:00
Dmitry S
fb1332d529 (core) Fix bug with renaming when a formula uses a local name for a user table.
Summary:
When a formula used a local variable referring to a user table (which is
a global name), e.g. `myvar = MyTable; myvar.lookupOne(...)`, renaming
logic mistakenly used the inferred name (`MyTable`) in places where the
actual variable name (`myvar`) should have been used.

Additionally, we should not rename local variables, even if they match a
global name.

This fixes both issues.

Test Plan: Added a test case that checks that local variables aren't renamed.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3846
2023-04-02 22:57:38 -04:00
Jarosław Sadziński
101450262c (core) Support for $ syntax in ACL rules
Summary: Adding support for the "$" syntax in ACL rules.

Test Plan: Updated

Reviewers: georgegevoian, dsagal

Reviewed By: georgegevoian, dsagal

Differential Revision: https://phab.getgrist.com/D3692
2022-11-09 16:33:11 +01:00
Alex Hall
49cb51bac5 (core) Error explanations from friendly-traceback
Summary: Extend formula error messages with explanations from https://github.com/friendly-traceback/friendly-traceback. Only for Python 3.

Test Plan: Updated several Python tests. In general, these require separate branches for Python 2 and 3.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3542
2022-08-12 19:45:00 +02:00
Alex Hall
b416a5c4b1 (core) Fix error when updating summary table formulas after rename
Summary:
Formulas in summary tables were being associated with the source table for automatic updating. When a table/column was renamed such that the formula needed to update to match, it would look for a column with the same colId but in the source table. Such a column might not exist which would lead to an error, or if it existed then the update would be wrong.

This association was created while building formulas to display in the code view in a nested `_Summary` class, it didn't need to exist at all. So this diff simply prevents the association from being created.

User report and discussion: https://grist.slack.com/archives/C0234CPPXPA/p1659717322297019

Test Plan: Extended `TestSummary.test_table_rename` Python test.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3568
2022-08-11 12:51:11 +02:00
Alex Hall
938928f1b9 (core) Fix renaming columns when iterating over Table.all in comprehension
Summary:
Comprehensions iterating over `Table.all` like `[foo.bar for foo in Table.all]` led to an error when renaming the column `bar`. This diff fixes that so that renaming `bar` does the same thing as for a comprehension over `Table.lookupRecords()`. Note that `next(foo for foo in Table.all).bar` is still not supported, as the same is not supported for `Table.lookupRecords()` either.

Discussion: https://grist.slack.com/archives/C069RUP71/p1658360276762949

Test Plan: Parametrised existing Python test to test the same thing for both `all` and `lookupRecords`

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3538
2022-07-21 18:30:16 +02:00
Alex Hall
29fb3360b6 (core) Update attributes of Table.all when column is renamed
Summary: Adds an InferenceTip which treats `Table.all` similarly to `Table.lookupRecords(...)`, so that `Table.all.foo` is changed to `Table.all.bar` when the column `foo` is renamed to `bar`.

Test Plan: Extended test for the `lookupRecords` case.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3521
2022-07-14 12:08:38 +02:00
Alex Hall
c5ebd7db3d (core) Add PEEK() function to bypass circular dependencies
Summary:
Adds a Python function `PEEK()` for use in formulas which temporarily sets a new attribute `Engine._peeking` which disables the `_use_node` method, preventing dependency tracking and allowing the given expression to use outdated values. This allows circumventing circular reference errors. It's particularly meant for trigger formulas although it works in normal formulas as well. The expression is wrapped in a `lambda` by `codebuilder` for lazy evaluation.

Discussion: https://grist.slack.com/archives/C0234CPPXPA/p1653571024031359

Test Plan: Added a Python unit test for circular trigger formulas using PEEK.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3453
2022-06-02 19:50:14 +02:00
Alex Hall
fb575a8b7e (core) Ensure formulas return something and don't assign to attributes of rec
Summary: To help with mistakes in formulas, forbid assigning to attributes of `rec` (e.g. `$foo = 1` which should probably be `==`) and ensure that there is at least one `return` in the formula (after maybe adding an implicit one at the end).

Test Plan: Extended Python unit test, updated tests which were missing return.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3439
2022-05-23 21:06:18 +02:00
Alex Hall
52fd28815e (core) Raise syntax errors that Python can format nicely to show the location
Summary: Update _create_syntax_error_code to raise an error with similar arguments to the real arguments it already has, with our modifications.

Test Plan: Updated python unit tests

Reviewers: jarek, dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3040
2021-09-24 15:07:09 +02:00
Alex Hall
305b133c59 (core) Remaining Python 3 compatibility changes
Summary: Biggest change is turning everything to unicode

Test Plan: The tests

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2875
2021-06-25 12:00:58 +02:00
Alex Hall
16f297a250 (core) Simple Python 3 compatibility changes
Summary: Changes that move towards python 3 compatibility that are easy to review without much thought

Test Plan: The tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2873
2021-06-22 17:13:17 +02:00
Paul Fitzpatrick
b82eec714a (core) move data engine code to core
Summary:
this moves sandbox/grist to core, and adds a requirements.txt
file for reconstructing the content of sandbox/thirdparty.

Test Plan:
existing tests pass.
Tested core functionality manually.  Tested docker build manually.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2563
2020-07-29 08:57:25 -04:00