Commit Graph

16 Commits

Author SHA1 Message Date
Alex Hall
1ed4ca61ea (core) Account for timezone in TODAY()
Summary: return NOW(tz=tz).date()

Test Plan: None, curious to see if this fixes test_time_defaults when run near midnight.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3079
2021-10-21 18:27:35 +02:00
Alex Hall
0e5c2bee59 (core) Add CellError to explain when a formula refers to a cell with an error value
Summary:
get_cell_value wraps RaisedException with CellError to expand the error message for the user.

This is still pretty conceptual, the comments explain some things to think about, but it works and is an improvement.

Test Plan: Updated Python unit tests

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2928
2021-07-30 22:58:45 +02:00
Alex Hall
04e5d90f86 (core) Barely working reference lists in frontend
Summary:
This makes it possible to set the type of a column to ReferenceList, but the UI is terrible

ReferenceList.ts is a mishmash of ChoiceList and Reference that sort of works but something about the CSS is clearly broken

ReferenceListEditor is just a text editor, you have to type in a JSON array of row IDs. Ignore the value that's present when you start editing. I can maybe try mashing together ReferenceEditor and ChoiceListEditor but it doesn't seem wise.
I think @georgegevoian should take over here. Reviewing the diff as it is to check for obvious issues is probably good but I don't think it's worth trying to land/merge anything.

Test Plan: none

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: georgegevoian

Differential Revision: https://phab.getgrist.com/D2914
2021-07-23 18:41:44 +02:00
Alex Hall
f5981606e1 (core) Make CONTAINS a function for consistency with mkpydocs etc.
Summary: Having CONTAINS be a class is a pain, undoing that mistake now

Test Plan: none needed

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2929
2021-07-21 13:18:23 +02:00
Alex Hall
f7a9638992 (core) CONTAINS() and summarising by ChoiceList columns with flattening
Summary:
Added CONTAINS 'function' which can be used in lookups

Changed LookupMapColumn._row_key_map to use right=set so one row can have many keys when CONTAINS is used.

Use CONTAINS to implement group column in summary table, while helper column in source table can reference and create multiple rows in summary table, especially when summarising by ChoiceList columns.

Use itertools.product to generate all combinations of lookup keys and groupby values.

cleanup

Test Plan: Added python unit tests.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: paulfitz, dsagal

Differential Revision: https://phab.getgrist.com/D2900
2021-07-19 16:35:35 +02:00
Alex Hall
305b133c59 (core) Remaining Python 3 compatibility changes
Summary: Biggest change is turning everything to unicode

Test Plan: The tests

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2875
2021-06-25 12:00:58 +02:00
Alex Hall
16f297a250 (core) Simple Python 3 compatibility changes
Summary: Changes that move towards python 3 compatibility that are easy to review without much thought

Test Plan: The tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2873
2021-06-22 17:13:17 +02:00
Alex Hall
8a940676e9 (core) Generic tools for recording pycalls, deterministic mode.
Summary:
Replaces https://phab.getgrist.com/D2854

Refactoring of NSandbox:
- Simplify arguments to NSandbox.spawn. Only half the arguments were used depending on the flavour, adding a layer of confusion.
- Ensure the same environment variables are passed to both flavours of sandbox
- Simplify passing down environment variables.

Implement deterministic mode with libfaketime and a seeded random instance.
- Include static prebuilt libfaketime.so.1, may need another solution in future for other platforms.

Recording pycalls:
- Add script recordDocumentPyCalls.js to open a single document outside of tests.
- Refactor out recordPyCalls.ts to support various uses.
- Add afterEach hook to save all pycalls from server tests under $PYCALLS_DIR
- Make docTools usable without mocha.
- Add useLocalDoc and loadLocalDoc for loading non-fixture documents

Test Plan:
Made a document with formulas NOW() and UUID()
Compare two document openings in normal mode:

    diff <(test/recordDocumentPyCalls.js samples/d4W6NrzCMNVSVD6nWgNrGC.grist /dev/stdout) \
         <(test/recordDocumentPyCalls.js samples/d4W6NrzCMNVSVD6nWgNrGC.grist /dev/stdout)

Output:

    <                 1623407499.58132,
    ---
    >                 1623407499.60376,
    1195c1195
    <               "B": "bd2487f6-63c9-4f02-bbbc-5c0d674a2dc6"
    ---
    >               "B": "22e1a4fd-297f-4b86-91a2-bc42cc6da4b2"

`export DETERMINISTIC_MODE=1` and repeat. diff is empty!

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2857
2021-06-15 20:58:05 +02:00
Paul Fitzpatrick
0c5f7cf0a7 (core) add SELF_HYPERLINK() function for generating links to the current document
Summary:
 * Adds a `SELF_HYPERLINK()` python function, with optional keyword arguments to set a label, the page, and link parameters.
 * Adds a `UUID()` python function, since using python's uuid.uuidv4 hits a problem accessing /dev/urandom in the sandbox.  UUID makes no particular quality claims since it doesn't use an audited implementation.  A difficult to guess code is convenient for some use cases that `SELF_HYPERLINK()` enables.

The canonical URL for a document is mutable, but older versions generally forward.  So for implementation simplicity the document url is passed it on sandbox creation and remains fixed throughout the lifetime of the sandbox.  This could and should be improved in future.

The URL is passed into the sandbox as a `DOC_URL` environment variable.

The code for creating the URL is factored out of `Notifier.ts`. Since the url is a function of the organization as well as the document, some rejiggering is needed to make that information available to DocManager.

On document imports, the new document is registered in the database slightly earlier now, in order to keep the procedure for constructing the URL in different starting conditions more homogeneous.

Test Plan: updated test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2759
2021-03-18 19:37:07 -04:00
Dmitry S
ba9959b6af (core) Fix broken markdown link in the doc-comment of VLOOKUP
Test Plan: No code changes

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2752
2021-03-09 18:00:59 -05:00
Dmitry S
2b71cce88b (core) Fix CONCAT function that was breaking on presence of non-ascii characters
Summary: This was the only occurrence of the unicode() function that I could find.

Test Plan: Added a doctest case.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2750
2021-03-09 16:27:48 -05:00
Dmitry S
003029bf8a (core) Improve suggestions for formula autocomplete
Summary:
- Make suggestions less case-sensitive (not entirely case-insensitive, but
  allow top-level suggestions to match in all-lowercase)
- Add function signatures to suggestions for Grist functions.
- Excel-like functions that are present but not implemented are no longer
  offered as suggestions.

Test Plan:
Added a test case on python side, and a browser test case for how suggestions
are rendered and inserted.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2608
2020-09-12 00:21:29 -04:00
Dmitry S
2fbd3f1706 (core) Fix lookups in default formulas
Test Plan: TODO

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2603
2020-09-10 18:06:06 -04:00
Dmitry S
7a8debae16 (core) Improve object serialization, to help get RECORD data to Custom Widgets.
Summary:
- Change RECORD's dates_as_str default to False.
- Reimplement objtype encode_object/decode_object with less machinery.
- Implement encoding of dicts (with string keys).
- Make lists and dicts encode values recursively.
- Implement encoding/decoding in the client
- Decode automatically in plugins' fetchSelectedTable/Record, with an option to skip.

Test Plan: Tested manually, not sure what tests may be affected yet.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2593
2020-08-21 18:33:28 -04:00
Dmitry S
4e11b6a922 (core) Implement RECORD function that converts Records to Python dictionaries
Summary:
Examining a Record is currently difficult, because its columns are hard to
list (and to use), and CircularRef errors hard to avoid. The RECORD function
takes care of this mess to return a simple dictionary of values.

- Supports dates_as_iso=False flag to turn off the translation of date/datetime
  objects to strings.
- Supports expand_refs=True flag to apply RECORD() to encountered values of
  type Record, for a single level of nesting.

Test Plan: Added a unittest for RECORD()

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2582
2020-08-12 13:31:57 -04:00
Paul Fitzpatrick
b82eec714a (core) move data engine code to core
Summary:
this moves sandbox/grist to core, and adds a requirements.txt
file for reconstructing the content of sandbox/thirdparty.

Test Plan:
existing tests pass.
Tested core functionality manually.  Tested docker build manually.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2563
2020-07-29 08:57:25 -04:00