Commit Graph

10 Commits

Author SHA1 Message Date
Alex Hall
d3dc910784 (core) sys.setdefaultencoding('utf8')
Summary: sys.setdefaultencoding('utf8')

Test Plan: Will test against user documents to check for changes

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2858
2021-06-16 20:44:25 +02:00
Paul Fitzpatrick
d0d3d3d0c9 (core) discount indirect changes for access control purposes
Summary:
This diff discounts indirect changes for access control purposes.  A UserAction that updates a cell A, which in turn causes changes in other dependent cells, will be considered a change to cell A for access control purposes.

The `engine.apply_user_actions` method now returns a `direct` array, with a boolean for each `stored` action, set to `true` if the action is attributed to the user or `false` if it is attributed to the engine.  `GranularAccess` ignores actions attributed to the engine when checking for edit rights.

Subtleties:
 * Removal of references to a removed row are considered direct changes.
 * Doesn't play well with undos as yet.  An action that indirectly modifies a cell the user doesn't have rights to may succeed, but it will not be reversible.

Test Plan: added tests, updated tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2806
2021-05-12 11:26:21 -04:00
Dmitry S
ec023a3ba6 (core) Fix another cause of inconsistency that can be triggered by bad DocActions.
Summary:
An incorrect DocAction (as possible from an Undo of a non-last action)
could cause RemoveRecord on an already missing record. This used to
create an invalid undo, and wreak havoc when a series of DocActions
later fails and needs to be reverted.

To fix, consider RemoveRecord of a missing record to be a no-op.

Test Plan: Includes a new test case that triggers the problem.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2717
2021-01-28 10:21:58 -05:00
Dmitry S
14cdd47675 (core) When checking for metadata consistency, check for stray column records too
Summary:
Currently, an undo of a non-last action can leave the doc in an inconsistent
state. For example, it may remove a table, but fail to remove all columns of
it from metadata. We normally check that schema corresponds to metadata, but
stray columns were not visible to this check, and instead caused later table
additions to fail.

This diff fixes the check to fail the action that causes stray columns, and
to restore the doc to a consistent state.

Note that this only handles schema-metadata inconsistencies, but an undo of a
non-last action can easily create other surprises.

Test Plan: Added a test case that triggered inconsistency before, and now triggers a failed undo.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2715
2021-01-27 18:10:11 -05:00
Dmitry S
9f806de64b (core) Allow using negative rowIds to add records and refer to them in Reference values.
Summary:
- When adding records, negative rowIds may now be specified. They'll be replaced by proper IDs.
- If these negative IDs are used in Reference columns in subsequent actions in
  the same bundle of UserActions, they get replaced with the proper rowIds.
- Use this to sync ACLResources and ACLRules from UI in a single batch of UserActions.
- Resolve the TODOs in GranularAccess test, to no longer need to guess resource rowIds.

Test Plan: Added a python unittest for mapping IDs; updated browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2691
2020-12-15 16:39:56 -05:00
Dmitry S
6b582b9ace (core) Remove the old attempt at ACLs implemented in Python.
Summary:
The new plans for granular access control are different and handled by
node.js. Some of the same tables will be reused, of which we never made
real use before except for expecting certain specific initial records.

This diff removes the old logic, replacing it with a stub that satisfies
the interface expected by other code.

It also removes several unused UserActions: AddUser/RemoveUser/
AddInstance/RemoveInstance.

Test Plan: Existing tests should pass.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2662
2020-11-12 09:35:08 -05:00
Dmitry S
e2226c3ab7 (core) Store formula values in DB, and include them into .stored/.undo fields of actions.
Summary:
- Introduce a new SQLiteDB migration, which adds DB columns for formula columns
- Newly added columns have the special ['P'] (pending) value in them
  (in order to show the usual "Loading..." on the first load that triggers the migration)
- Calculated values are added to .stored/.undo fields of user actions.
- Various changes made in the sandbox to include .stored/.undo in the right order.
- OnDemand tables ignore stored formula columns, replacing them with special SQL as before
- In particular, converting to OnDemand table leaves stale values in those
  columns, we should maybe clean those out.

Some tweaks on the side:
- Allow overriding chai assertion truncateThreshold with CHAI_TRUNCATE_THRESHOLD
- Rebuild python automatically in watch mode

Test Plan: Fixed various tests, updated some fixtures. Many python tests that check actions needed adjustments because actions moved from .stored to .undo. Some checks added to catch situations previously only caught in browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2645
2020-11-04 16:45:47 -05:00
Dmitry S
003029bf8a (core) Improve suggestions for formula autocomplete
Summary:
- Make suggestions less case-sensitive (not entirely case-insensitive, but
  allow top-level suggestions to match in all-lowercase)
- Add function signatures to suggestions for Grist functions.
- Excel-like functions that are present but not implemented are no longer
  offered as suggestions.

Test Plan:
Added a test case on python side, and a browser test case for how suggestions
are rendered and inserted.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2608
2020-09-12 00:21:29 -04:00
Dmitry S
2fbd3f1706 (core) Fix lookups in default formulas
Test Plan: TODO

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2603
2020-09-10 18:06:06 -04:00
Paul Fitzpatrick
b82eec714a (core) move data engine code to core
Summary:
this moves sandbox/grist to core, and adds a requirements.txt
file for reconstructing the content of sandbox/thirdparty.

Test Plan:
existing tests pass.
Tested core functionality manually.  Tested docker build manually.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2563
2020-07-29 08:57:25 -04:00