Commit Graph

275 Commits

Author SHA1 Message Date
Cyprien P
4cd9a2d740 (core) Making side bars a bit more usable for narrow screen
Summary:
 - closes side bars when tapping content area
 - opens left panel when tapping the page name

Test Plan: Includes browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2705
2021-01-19 15:22:01 +01:00
Paul Fitzpatrick
3ad9b18ddf (core) allow a doc owner to test access as a different user
Summary:
This adds back-end support for query parameters `aclAsUser_` and
`aclAsUserId_` which, when either is present, direct Grist to
process granular access control rules from the point of view
of that user (specified by email or id respectively).

Some front end support is added, in the form of a tag that
shows up when in this mode, and a way to cancel the mode.
No friendly way to initiate the mode is offered yet.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2704
2021-01-15 18:45:57 -05:00
Dmitry S
d8e742aa0d (core) Add getAclResources method for making all tables/columns available when editing ACL rules
Summary:
The goal is that those who can edit ACL rules can create or change rules for
any resource, even if the rules block their own ability to see the resource.

Test Plan: Added a browser test, and a server test for who can call the new method.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2703
2021-01-14 13:43:55 -05:00
Cyprien P
ffe4a34335 (core) Adds current page name to the bottom bar [narrow screen]
Test Plan: Tested Manually, behave well also for long page name.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2702
2021-01-14 18:30:48 +01:00
Cyprien P
4ca1c09e35 (core) Adds scafoldings for the narrow screen layout
Summary:
This commit adds a bottom bar with 2 openers  button for each side panels.
It takes care of showing/hiding other openers when appropriate.
It also implements the new style for narrow screen with the side panels overlapping the main pane.

Remaining tasks:
  - adjust content of the breadcrumbs for narrow screen.
  - content of the bottom bar

Test Plan: Adds test to projects/PagePanels

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D2701
2021-01-13 16:13:44 +01:00
Paul Fitzpatrick
438f259687 (core) start reconciling forking with granular access
Summary:
This allows a fork to be made by a user if:
 * That user is an owner of the document being forked, or
 * That user has full read access to the document being forked.

The bulk of the diff is reorganization of how forking is done.  ActiveDoc.fork is now responsible for creating a fork, not just a docId/urlId for the fork. Since fork creation should not be limited to the doc worker hosting the trunk, a helper endpoint is added for placing the fork.

The change required sanitizing worker allocation a bit, and allowed session knowledge to be removed from HostedStorageManager.

Test Plan: Added test; existing tests pass.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2700
2021-01-12 14:08:49 -05:00
Paul Fitzpatrick
68a682f876 (core) add housekeeping endpoints for cleaning doc snapshots+state
Summary:
This adds endpoints that allow the support user to remove unlisted
snapshots for a document, and to remove all action history for
a document.

This does increase what the support user can do, but not in a way
that would be particularly valuable to attack.  It would have some
destructive value, for removing history (removing unlisted
snapshots doesn't impact the user, by contrast).

This would simplify some maintenance operations.

Test Plan: added test for snapshots; tested states manually

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2699
2021-01-05 10:31:14 -05:00
Dmitry S
5deac68315 (core) Update ACL resources/rules when tables/columns get renamed
Summary:
- Placed rule-updating functions in acl.py.
- Reset UI when rules update externally, or alert the user to reset if there
  are pending local changes.
- Removed some unused and distracting bits from client-side DocModel.

A few improvements related to poor error handling:
- In case of missing DocActions (tickled by broken ACL rule handling), don't
  add to confusion by attempting to process bad actions
- In case of missing attributes in ACL formulas, return undefined rather than
  fail; the latter creates more problems.
- In case in invalid rules, fail rather than skip; this feels more correct now
  that we have error checking and recovery option, and helps avoid invalid rules.
- Prevent saving invalid rules with an empty ACL formula.
- Fix bug with rule positions.

Test Plan: Added a python and browser test for table/column renames.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2698
2021-01-04 22:03:01 -05:00
Dmitry S
d6d1eb217f (core) One more phase of ACL UI revision.
Summary:
- Add ACLColumnList widget for a list of column IDs.
- Replace autocomplete widgets with simpler dropdowns.
- Add select dropdown for the Attribute of UserAttribute rules.
- Switch formula to use ACE editor.
- Factor out customized completion logic from AceEditor.js into a separate file.
- Implement completions for ACL formulas.
- Collect ACL UI files in app/client/aclui

Test Plan: Updated test case, some behavior (like formula autocomplete) only tested manually.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2697
2020-12-22 22:18:12 -05:00
Dmitry S
4ad84f44a7 (core) Improve the UI for ACL rules.
Summary:
- Add headers to tables.
- Change styles to reduce boxes-within-boxes.
- Add validation of table and column IDs, both in UI and on server when saving rules.
- Add autocomplete for tables/columns used for UserAttribute rules.
- Add a fancy widget to set permission bits.

Test Plan: Updated browser test for new UI, added a test case for user attributes.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2695
2020-12-22 13:40:52 -05:00
Paul Fitzpatrick
d5b00f5169 (core) add explicit doc and inventory creation step
Summary:
Currently, if a document is created by importing a file, inventory
creation is a little haphazard - it works, but triggers a
"surprise" message.  This diff makes initialization of inventory
explicit, so that surprise messages shouldn't happen during
document creation.

Test Plan: manual

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2696
2020-12-21 11:13:03 -05:00
Paul Fitzpatrick
24e76b4abc (core) add endpoints for clearing snapshots and actions
Summary:
This adds a snapshots/remove and states/remove endpoint, primarily
for maintenance work rather than for the end user.  If some secret
gets into document history, it is useful to be able to purge it
in an orderly way.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2694
2020-12-18 13:32:31 -05:00
Paul Fitzpatrick
b1c4af4ee9 (core) correctly delete pruned document versions
Summary:
After switch to using an inventory file, old document versions were
not in fact being pruned.  This corrects that and adds a test
that fails with the previous implementation.

The pruner was operating correctly, but was being applied to an
inventory list rather than s3 directly - and the inventory list
did not pass through version removals to s3.

This fix will leave a stock of undeleted versions that can
be eliminated by an external script (there are alternatives
but that seems simplest overall).

Test Plan: updated test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2693
2020-12-16 15:11:15 -05:00
Dmitry S
12a7059bde (core) Fix bug with pasting in the presence of link-filtering.
Summary:
The bug manifested when multiple grid cells were selected, and then user
selected a record in another section that caused the grid to show a different
set of rows. Paste would then go into multiple rows even when they are not
visibly selected.

The test includes some porting of hacks from old browser tests to support
copy-pasting. In particular gu.sendKeys() is a useful alternative to
driver.find('body').sendKeys() which we've been using to work around
driver.sendKeys() limitations, but which apparently causes flakiness with
focus.

Test Plan: Browser test reproduces the bug before the fix.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2692
2020-12-16 14:50:39 -05:00
Dmitry S
9f806de64b (core) Allow using negative rowIds to add records and refer to them in Reference values.
Summary:
- When adding records, negative rowIds may now be specified. They'll be replaced by proper IDs.
- If these negative IDs are used in Reference columns in subsequent actions in
  the same bundle of UserActions, they get replaced with the proper rowIds.
- Use this to sync ACLResources and ACLRules from UI in a single batch of UserActions.
- Resolve the TODOs in GranularAccess test, to no longer need to guess resource rowIds.

Test Plan: Added a python unittest for mapping IDs; updated browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2691
2020-12-15 16:39:56 -05:00
Paul Fitzpatrick
b2fabb0ebc (core) respect table wildcard in granular acl rules for censorship
Summary:
The client relies on metadata tables for laying out pages and sections.
These tables are filtered according to what tables the user has access
to, in a crude way.  This diff updates the logic to at least support
the table wildcard.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2690
2020-12-15 09:51:11 -05:00
Dmitry S
de35be6b0a (core) Checks that an ACL formula can be parsed, and prevent saving unparsable ACL rules.
Summary:
- Fix error-handling in bundleActions(), and wait for the full bundle to complete.
  (The omissions here were making it impossibly to react to errors from inside bundleActions())
- Catch problematic rules early enough to undo them, by trying out ruleCollection.update()
  on updated rules before the updates are applied.
- Added checkAclFormula() call to DocComm that checks parsing and compiling
  formula, and reports errors.
- In UI, prevent saving if any aclFormulas are invalid, or while waiting for the to get checked.

- Also fixed some lint errors

Test Plan: Added a test case of error reporting in ACL formulas.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2689
2020-12-15 09:43:37 -05:00
Paul Fitzpatrick
3b3ae87ade (core) implement a safe mode for opening documents with rule problems
Summary:
Adds an "enter safe mode" option and explanation in modal that appears when a document fails to load, if user is owner. If "enter safe mode" is selected, document is reloaded on server in a special mode. Currently, the only difference is that if the acl rules fail to load, they are replaced with a fallback that grants full access to owners and no access to anyone else. An extra tag is shown to mark the document as safe mode, with an "x" for cancelling safe mode.

There are other ways a document could fail to load than just acl rules, so this is just a start.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2686
2020-12-14 13:04:13 -05:00
Dmitry S
02ed4c59a0 (core) On welcome form pages, prevent accidental multiple submissions.
Summary:
Disable the submit button while a form submission is pending.

(We don't move to the next page without waiting since we get the page to
redirect to from the server.)

Test Plan: Tested manually by pausing the server and trying to submit multiple times.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2688
2020-12-11 22:42:20 -05:00
Paul Fitzpatrick
e5c24eb5ea (core) revamp user attribute handling
Summary:
This changes how user attributes are loaded.  They are now loaded
directly from sqlite, with per-session caching.  Optimizations
considered but not addressed yet are (1) adding indexes to user attribute
tables and (2) swapping in a thinner sqlite wrapper.

The main benefit of this diff is that changes to user attribute
tables now work.  Clients whose user attributes are not changed
see no effect; clients whose user attributes have changed have
their document reloaded.

For the purposes of testing, the diff includes a tweak to
GristWSConnection to be "sticky" to a specific user when reloading
(and support machinery on the server side to honor that).  Until
now, if a GristWSConnection reloads, it uses whatever the current
default user is in the cookie-based session, which can change.
This was complicating a test where multiple users were accessing
the same document via different clients with occasional document
reloads.

Code for updating when schema or rule changes happen is moved
around but not improved in any meaningful way in this diff.

Test Plan: existing tests pass; extended test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2685
2020-12-11 15:15:35 -05:00
Dmitry S
e5d41c72dc (core) Fix JS error when switching from DocHistory to ActionLog too quickly.
Summary:
Also added a Chrome option to tests to suppress popup to save passwords,
and unified setting of Chrome options in one shared file.

Test Plan: Tested locally, reproduced the problem by adding a delay in BaseAPI.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2687
2020-12-11 14:33:29 -05:00
Dmitry S
b3a57e3b5c (core) Be more careful with avoiding actions which don't change encoded values
Summary:
- For comparing for equality of encoding, do better at approximating what's equal in JSON.
- Fix loading of "RaisedException" values so that they can match an equivalent
  exception raised when the formula is re-evaluated.

Test Plan: Added another column to the test that verifies the Calculate action.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2682
2020-12-10 09:09:45 -05:00
Dmitry S
4f263fc7ec (core) Prevent stray 'compare' param from appearing in some snapshot links
Summary:
Some slight difference appeared, possibly during rebase, about including
compare parameters in URLs.

Test Plan: Browser test was failing, now passes

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2683
2020-12-09 18:21:44 -05:00
Dmitry S
92224084e4 (core) Improve snapshot listing, and add compare snapshot links
Summary:
- When viewing a snapshot, list all snapshots for a doc, highlighting the current one.
- Include links in per-snapshot menu to compare-to-current and compare-to-previous.
- Compare links include "beta" tags.
- Set order of comparison to have older on the left, and newer on the right.

Test Plan: Moved out DocHistory test from Snapshots, and added some test cases.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2681
2020-12-09 16:48:19 -05:00
Paul Fitzpatrick
8f023a6446 (core) implement authorization via query parameter
Summary:
This adds any parameters in a document url whose key ends in '_'
into a `user.Link` object available in access control formulas
and in setting up characteristic tables.

This allows, for example, sending links to a document that contain
a hard-to-guess token, and having that link grant access to a
controlled part of the document (invoices for a specific customer
for example).

A `user.Origin` field is also added, set during rest api calls,
but is only tested manually at this point.  It could be elaborated
for embedding use-cases.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2680
2020-12-09 09:48:06 -05:00
Paul Fitzpatrick
131fbbdb92 (core) check row-level permissions on incoming actions
Summary:
This improves support for access control on document modifications.  It adds:

   * Checking of create/remove/update access for row-level changes.
   * Use of `newRec` variable in formulas.

It is now possible to have distinct clients with read+write access to different rows of the same table.

This is another incremental step.  There are deficiencies in actions that include schema changes, and many other lacunae. But the overall flow is taking shape.

Access control is done at the DocAction level, requiring the sandbox to process the UserActions, and then be reverted if the action proves unlawful.  This could be optimized away in many simple and important cases, but I'm not sure it is possible to avoid in general.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2677
2020-12-07 16:59:28 -05:00
Dmitry S
8c788005c3 (core) Implement much of the general AccessRules UI.
Summary:
- Factored out ACLRuleCollection into its own file, and use for building UI.
- Moved AccessRules out of UserManager to a page linked from left panel.
- Changed default RulePart to be the last part of a rule for simpler code.
- Implemented much of the UI for adding/deleting rules.
  - For now, editing the ACLFormula and Permissions is done using text inputs.
- Implemented saving rules by syncing a bundle of them.
- Fixed DocData to clean up action bundle in case of an early error.

Test Plan: WIP planning to add some new browser tests for the UI

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2678
2020-12-07 14:48:41 -05:00
Paul Fitzpatrick
3519d0efce (core) attribute Calculate to grist rather than user
Summary:
when a document is opened, and Calculate results in
a change, that change is not attributed to 'grist' rather than
to the user's email.

Some minor tweaks included to freshen some related dev scripts.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2679
2020-12-07 12:46:17 -05:00
Dmitry S
0289e3ea17 (core) Fix issue with spurious changes produced by Calculate action.
Summary:
- Replace unicode strings with byte strings when decoding values in sandbox.
- Columns that rely on float values should derive from NumericColumn, so
  that set() ensures that a float is stored even if loading an int.
- Parse unmarshallable values (['U']) into an object that can be encoded
  back to the same value (rather than info a RaisedException).
- Compare NaN's as equal for deciding whether a change is a no-op.

Unrelated:
- Removed a tiny bit of unhelpful logging

Test Plan:
Added a test case that reproduces several causes of Calculate
discrepancies by loading various values into various types of formula columns.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2676
2020-12-03 14:10:26 -05:00
Paul Fitzpatrick
0e2deecc55 (core) implement cleaner row-level access control for outgoing messages
Summary:
This implements row-level access control for outgoing messages, replacing the document reloading placeholder that was there before.

 * Prior to broadcasting messages, GranularAccess is notified of actions+undo.
 * While broadcasting messages to different sessions, if we find we need row level access control information, rows before and after the change are reconstructed.
 * Messages are rewritten if rows that were previously forbidden are now allowed, and vice versa.

The diff is somewhat under-tested and under-optimized. Next step would be to implement row-level access control for incoming actions, which may result in some rejiggering of the code from this diff to avoid duplication of effort under some conditions.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2670
2020-11-30 16:28:33 -05:00
Paul Fitzpatrick
c1c17bf54e (core) distinguish open public documents from listing them
Summary:
getOrgWorkspaces and getWorkspaces had an unintended feature where
if a user had access to a workspace, they could list all publically
shared documents within that workspace.  This diff stops considering
resources shared with everyone@ when listing orgs or workspaces.
Resources shared with anon@ remain listed - this is how the example
workspace operates.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2671
2020-11-27 09:56:25 -05:00
Dmitry S
32f3d03c3d (core) Mark 'Compare to Original' menu item with the 'beta' tag
Test Plan: Manually checked the tweaked look.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2674
2020-11-27 08:10:17 -05:00
Dmitry S
840f46d051 (core) Fix typings in DocApi causing a build failure in core.
Test Plan: Core test now passes.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2675
2020-11-27 08:09:57 -05:00
Dmitry S
9a7a42bc59 (core) Fix "Copy Link" in UserManager when its opened from a DocMenu page.
Test Plan: Added a check to the test case verifying the copied link.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2673
2020-11-26 22:30:01 -05:00
Dmitry S
5172cae1c2 (core) Fix a bug that occurs after remaing a table containing a RefList column.
Summary:
This can happen after "Detach" is used on a summary table.
Includes a unittest reproducing the problem case.

Test Plan: Included unittest fails without the one-line tweak.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2672
2020-11-26 22:29:37 -05:00
Dmitry S
4539521dff (core) Convert a few widgets to typescript and grainjs.
Summary:
No behavior changes.
Diff includes an intermediate commit with only renames, for easier review.

Test Plan: Existing tests should pass.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2669
2020-11-23 09:49:55 -05:00
Dmitry S
f24a82e8d4 (core) Some cleanup: remove old unused modules.
Summary:
- Remove modules related to old login / profile that we don't plan to bring back.
- Remove old unused DocListModel.
- Remove ext* tests that have been skipped and don't work.
- Remove old ModalDialog, and switch its one remaining usage to the newer way.

Test Plan: All tests should pass, and as many as before.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2668
2020-11-20 11:23:20 -05:00
Dmitry S
2e22966289 (core) Revamp attachment editor + preview UI, and support more types, including PDFs.
Summary:
- New UI for the modal look mostly following the design prepared previously.
- Use <object> for rendering PDFs (on a Mac works on Firefox, Chrome, Safari; needs checking on Windows)
- While we are at it, use <video> and <audio> for relevant files (object would
  render them too, but without the option to disable autoplay).
- Use <object> for unknown types, except for text/html (unsafe) and other text
  types (need more work to render well).
- Fix skipping save on Escape or when attachments are unsaved (previously a
  noop action was emitted, creating surprises with undo).
- Display extension for files without preview, both in-cell and in the modal.
- Replace tiny "eye" icon to preview particular attachment with double-clicking.
- As an accidental feature, a particular attachment can be previewed by typing 1, 2, 3, etc into cell.
- Renamed PreviewsWidget/PreviewModel to AttachmentsWidget/AttachmentsEditor.

Test Plan: Unified old and new tests for attachments, added new test cases.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2667
2020-11-19 21:30:09 -05:00
Paul Fitzpatrick
c387fc4bce (core) hide long sequences of unchanged rows in diffs
Summary:
It can be hard to find changes, even when highlighted, in a table with many rows.  This diff replaces long sequences of unchanged rows with a row containing "..."s.

With daff, I found that it is important to do this for sequences of unchanged columns also, but not tackling that yet.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2666
2020-11-19 18:19:54 -05:00
Dmitry S
bc3a472324 (core) Implement new representation of ACL rules.
Summary:
- Added fields to _grist_ACLRules for the new Granular ACL representation
- Include a corresponding migration.

- Added ACLPermissions module with merging PermissionSets and converting to/from string.
- Implemented parsing of ACL formulas and compiling them into JS functions.
- Add automatic parsing of ACL formulas when ACLRules are added or updated.
- Convert GranularAccess to load and interpret new-style rules.
- Convert ACL UI to load and save new-style rules.

For now, no attempt to do anything better on the server or UI side, only to
reproduce previous behavior.

Test Plan: Added unittests for new files; fixed those for existing files.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2664
2020-11-18 08:58:03 -05:00
Dmitry S
c042935c58 (core) Fix bug where a newly-created doc allows undo, which breaks it
Test Plan: Added a check to an existing test.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2665
2020-11-17 10:12:23 -05:00
Paul Fitzpatrick
ab01ce495d (core) make ValueFormatter.format honor its return type
Summary: make ValueFormatter.format honor its return type

Test Plan: existing tests pass

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D2663
2020-11-12 15:19:38 -05:00
Paul Fitzpatrick
f1842cd89e (core) tolerate table renames when displaying differences
Summary:
This makes data diff rendering robust to changes in the names of tables.
It does not yet show information about those changes, but at least it
won't fail to show table content changes.

Added a missing case to ActionSummary concatenation that came up in
testing.

Test Plan: added test, updated test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2661
2020-11-12 10:55:15 -05:00
Dmitry S
6b582b9ace (core) Remove the old attempt at ACLs implemented in Python.
Summary:
The new plans for granular access control are different and handled by
node.js. Some of the same tables will be reused, of which we never made
real use before except for expecting certain specific initial records.

This diff removes the old logic, replacing it with a stub that satisfies
the interface expected by other code.

It also removes several unused UserActions: AddUser/RemoveUser/
AddInstance/RemoveInstance.

Test Plan: Existing tests should pass.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2662
2020-11-12 09:35:08 -05:00
Dmitry S
5b2de988b5 (core) Perform migrations of Grist schema using only metadata tables when possible.
Summary:
Loading all user data to run a migration is risky (creates more than usual
memory pressure), and almost never needed (only one migration requires it).

This diff attempts to run migrations using only metadata (_grist_* tables),
but retries if the sandbox tells it that all data is needed.

The intent is for new migrations to avoid needing all data.

Test Plan: Added a somewhat contrived unittest.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2659
2020-11-11 19:21:40 -05:00
Paul Fitzpatrick
5a9fe0ea27 (core) show differences in card views when comparing documents
Summary:
This makes a small tweak to show cell and row changes in card views
and card list views, and adds a test for it.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2660
2020-11-11 17:25:38 -05:00
Paul Fitzpatrick
c67966775b (core) simplify document comparison code, and flesh out diff with local changes
Summary:
With recent changes to action history, we can now remove the temporary
`finalRowContent` field from change details, since all the information
we need is now in the ActionSummary.

We also now have more information about the state of the common ancestor,
which previously we could not get either from ActionSummary or from
`finalRowContent`. We take advantage of that to flesh out rendering
differences where there are some changes locally and some changes
remotely.

There's still a lot more to do, this is just one step.

I have added a link to the UI for viewing the comparison. I wouldn't
want to advertise that link until diffs are robust to name changes.

Test Plan: added test, updated tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2658
2020-11-11 15:49:16 -05:00
Dmitry S
2a592d8b4d (core) Automatically finalize action bundles when unrelated actions/bundles come in.
Summary:
Type conversions and formula tranforms wait for the user and bundle multiple
actions. When an unrelated action is done (e.g. adding a page widget or a
column), we want to finalize the transform before applying it.

The approach turns out fairly complicated. There is an implicit queue of
bundles (which we don't let grow beyond 2, as that's too abnormal). Bundles may
be finalized by a user clicking something, or by an unrelated action/bundle, or
(as before) by transform DOM getting disposed.

- Updated RecordLayout to use bundleActions() helper
- Added support for nesting bundleActions inside another bundle (needed for
  setting visibleCol during type change)
- In an unrelated tweak, when in debug-log in ActiveDoc, use a short representation of result.

Test Plan: Added a unittest for action bundling during type transform

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2655
2020-11-10 10:32:07 -05:00
Paul Fitzpatrick
e30d0fd5d0 (core) fix sync to s3 when doc is marked as dirty but proves to be clean
Summary:
This fixes a two problems:
 * A mistake in `KeyedMutex.runExclusive`.
 * Logic about saving a document to s3 when the document is found to match what is already there.

`HostedStorageManager.flushDoc` could get caught in a loop if a document was uploaded to s3 and then, without any change to it, marked as dirty.  Low level code would detect there was no change and skip the upload; but then the snapshotId could be unknown, causing an error and retries. This diff fixes that problem by discovering the snapshotId on downloads and tracking it. It also corrects a mutex problem that may have been creating the scenario. A small delay is added to `flushDoc` to mitigate the effect of similar problems in future. Exponential backoff would be good, but `flushDoc` is called in some situations where long delays would negatively impact worker shutdown or user work.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2654
2020-11-10 08:12:31 -05:00
Dmitry S
6d95418cc1 (core) Close previous example card in the rare cases when a second one might be triggered
Test Plan: Added a test case

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2656
2020-11-09 23:46:47 -05:00