Commit Graph

67 Commits

Author SHA1 Message Date
Paul Fitzpatrick
cf23a2d1ee (core) add GVISOR_LIMIT_MEMORY to cap memory available in sandbox
Summary:
This allows limiting the memory available to documents in the sandbox when gvisor is used. If memory limit is exceeded, we offer to open doc in recovery mode. Recovery mode is tweaked to open docs with tables in "ondemand" mode, which will generally take less memory and allow for deleting rows.

The limit is on the size of the virtual address space available to the sandbox (`RLIMIT_AS`), which in practice appears to function as one would want, and is the only practical option. There is a documented `RLIMIT_RSS` limit to `specifies the limit (in bytes) of the process's resident set (the number of virtual pages resident in RAM)` but this is no longer enforced by the kernel (neither the host nor gvisor).

When the sandbox runs out of memory, there are many ways it can fail. This diff catches all the ones I saw, but there could be more.

Test Plan: added tests

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3398
2022-05-18 14:26:27 -04:00
George Gevoian
2fd8a34ff8 (core) Move Notifier to /ext
Summary:
This makes it possible to configure a SendGrid-based Notifier
instance via a JSON configuration file.

Test Plan: Tested manually.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3432
2022-05-18 08:02:32 -07:00
George Gevoian
f48d579f64 (core) Add API endpoint to get site usage summary
Summary:
The summary includes a count of documents that are approaching
limits, in grace period, or delete-only. The endpoint is only accessible
to site owners, and is currently unused. A follow-up diff will add usage
banners to the site home page, which will use the response from the
endpoint to communicate usage information to owners.

Test Plan: Browser and server tests.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3420
2022-05-16 11:16:19 -07:00
George Gevoian
1e42871cc9 (core) Add attachment and data size usage
Summary:
Adds attachment and data size to the usage section of
the raw data page. Also makes in-document usage banners
update as user actions are applied, causing them to be
hidden/shown or updated based on the current state of
the document.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3395
2022-05-04 13:46:55 -07:00
Alex Hall
dc9e53edc8 (core) Update the current time in formulas automatically every hour
Summary: Adds a special user action `UpdateCurrentTime` which invalidates an internal engine dependency node that doesn't belong to any table but is 'used' by the `NOW()` function. Applies the action automatically every hour.

Test Plan: Added a Python test for the user action. Tested the interval periodically applying the action manually: {F43312}

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3389
2022-04-28 21:07:40 +02:00
George Gevoian
af5b3c9004 (core) Add document usage banners
Summary:
This also enables the new Usage section for all sites. Currently,
it shows metrics for document row count, but only if the user
has full document read access. Otherwise, a message about
insufficient access is shown.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3377
2022-04-25 08:14:52 -07:00
Alex Hall
02e69fb685 (core) Crudely show row count and limit in UI
Summary:
Add rowCount returned from sandbox when applying user actions to ActionGroup which is broadcast to clients.

Add rowCount to ActiveDoc and update it after applying user actions.

Add rowCount to OpenLocalDocResult using ActiveDoc value, to show when a client opens a doc before any user actions happen.

Add rowCount observable to DocPageModel which is set when the doc is opened and when action groups are received.

Add crude UI (commented out) in Tool.ts showing the row count and the limit in AppModel.currentFeatures. The actual UI doesn't have a place to go yet.

Followup tasks:

- Real, pretty UI
- Counts per table
- Keep count(s) secret from users with limited access?
- Data size indicator?
- Banner when close to or above limit
- Measure row counts outside of sandbox to avoid spoofing with formula
- Handle changes to the limit when the plan is changed or extra rows are purchased

Test Plan: Tested UI manually, including with free team site, opening a fresh doc, opening an initialised doc, adding rows, undoing, and changes from another tab. Automated tests seem like they should wait for a proper UI.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3318
2022-03-14 21:49:32 +02:00
Paul Fitzpatrick
accd640078 (core) add a user.SessionID value for trigger formulas and granular access rules
Summary:
This makes a `user.SessionID` value available in information about the user, for use with trigger formulas and granular access rules. The ID should be constant within a browser session for anonymous user. For logged in users it simply reflects their user id.

This ID makes it possible to write access rules and trigger formulas that allow different anonymous users to create, view, and edit their own records in a document.

For example, you could have a brain-storming document for puns, and allow anyone to add to it (without logging in), letting people edit their own records, but not showing the records to others until they are approved by a moderator. Without something like this, we could only let anonymous people add one field of a record, and not have a secure way to let them edit that field or others in the same record.

Also adds a `user.IsLoggedIn` flag in passing.

Test Plan: Added a test, updated tests. The test added is a mini-moderation doc, don't use it for real because it allows users to edit their entries after a moderator has approved them.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3273
2022-02-22 12:50:43 -05:00
Edward Betts
d6e0e1fee3 Correct spelling mistakes 2022-02-19 09:46:49 +00:00
Alex Hall
5d671bf0b3 (core) New type conversion in the backend
Summary: This is https://phab.getgrist.com/D3205 plus some changes (https://github.com/dsagal/grist/compare/type-convert...type-convert-server?expand=1) that move the conversion process to the backend. A new user action ConvertFromColumn uses `call_external` so that the data engine can delegate back to ActiveDoc. Code for creating formatters and parsers is significantly refactored so that most of the logic is in `common` and can be used in different ways.

Test Plan: The original diff adds plenty of tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3240
2022-02-04 20:28:13 +02:00
Paul Fitzpatrick
d8aacbe3b4 (core) AddOrUpdateRecord user action
Summary:
New user action as described in https://grist.quip.com/fZSrAnJKgO5j/Add-or-Update-Records-API, with options to allow most of the mentioned possible behaviours.

The Python code is due to Alex (as should be obvious from the u in behaviours).

Test Plan: Added a unit test.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3239
2022-02-03 16:22:51 -05:00
Alex Hall
faec8177ab (core) Use MetaTableData more
Summary:
Add more method overrides to MetaTableData for extra type safety.

Use MetaTableData, MetaRowRecord, and getMetaTable in more places.

Test Plan: Mostly it just has to compile. Tested manually that types are being checked more strictly now, e.g. by adding a typo to property names. Some type casting has also been removed.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3168
2021-12-07 17:09:58 +02:00
Jarosław Sadziński
53bdd6c8e1 (core) Exposing more descriptive errors from exports
Summary:
Exports used to show generic message on error.
Adding error description to the message.

Test Plan: Updated tests

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3157
2021-11-30 17:26:32 +01:00
Alex Hall
a64fb105e3 (core) Use GristObjCode in CellValue
Summary: Makes type checking a bit stronger

Test Plan: it just has to compile

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3065
2021-10-11 14:11:32 +02:00
Paul Fitzpatrick
d635c97686 (core) flesh out "View As" feature
Summary:
The users shown by the "View As" button are now drawn from more sources:
 * There are users the document is shared with. This has been rationalized, the behavior was somewhat erratic. If the user is not an owner of the document, the only user of this kind that will be listed is themselves.
 * There are users mentioned in any user attribute table keyed by Email. If name and access columns are present, those are respected, otherwise name is taken from email and access is set to "editors".
 * There are example users provided if there are not many other users available.

Test Plan: added and extended tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3045
2021-10-08 12:00:40 -04:00
Paul Fitzpatrick
07558dceba (core) avoid censorship for one client clobbering data for another client
Summary:
When filtering document updates to send to clients after a change,
censorship of individual cells was being applied to state shared
across the clients. This diff eliminates that shared state, and
extends testing of broadcasts to check different orderings.

Test Plan:
extends a test to tickle a reported bug, and gives
DocClients a knob to control message order needed to tickle
the bug reliably.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3064
2021-10-07 23:21:07 -04:00
Paul Fitzpatrick
d5a7fb23fe (core) apply SchemaEdit flag to metadata changes in general
Summary:
A user without SchemaEdit permission was able to reorder pages, since
this changes _grist_Pages, and that table was left under control of
regular access rules.  This diff tightens things up, to require
SchemaEdit for all metadata edits.  The one remaining exception is
_grist_Attachments, which needs some reworking to play well with
granular access.

Test Plan: extended test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3025
2021-09-16 13:36:20 -04:00
Paul Fitzpatrick
b3636b97e2 (core) Report memos consistently for blocked actions involving schema
Summary:
Currently actions blocked early because they could modify the
schema (e.g. changing formulas) do not report memo information
(comments in relevant rules).  This diff fixes that by using
more of the same code path in the two situations.  It also
adds information about what type of action was blocked to
error messages.

Test Plan: extended a test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2995
2021-08-24 08:03:58 -04:00
Paul Fitzpatrick
f53ab2cb30 (core) forbids edits when "view as" user is a viewer and access rules are permissive
Summary:
Currently, if access rules are set to allow edits unconditionally,
and an owner does "View As" a user who is a viewer only, they will
be allowed to make edits.  This catches that condition and adds a
test.

Test Plan: added test

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D2991
2021-08-20 16:04:28 -04:00
Alex Hall
7f1f8fc9e6 (core) Linking summary tables grouped by list columns
Summary:
Prefix keys of `LinkingState.filterColValues` with `_contains:` when the source column is a ChoiceList or ReferenceList.

This is parsed out to make a boolean `isContainsFilter` which is kept in each value of `QueryRefs.filterTuples` (previously `filterPairs`).

Then when converting back in `convertQueryFromRefs` we construct `Query.contains: {[colId: string]: boolean}`.

Finally `getFilterFunc` uses `Query.contains` to decide what kind of filtering to do.

This is not pretty, but the existing code is already very complex and it was hard to find something that wouldn't require touching loads of code just to make things compile.

Test Plan: Added a new nbrowser test and fixture, tests that selecting a source table by summary tables grouped by a choicelist column, non-list column, and both all filter the correct data.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2940
2021-08-10 20:41:24 +02:00
Paul Fitzpatrick
4a23b964ed (core) update read access for exceptional sessions
Summary:
Exceptional sessions had lost full read access to documents; this
restores it.  Exceptional sessions are used for system actions or
while creating documents.

Test Plan: added test

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D2966
2021-08-05 16:49:23 -04:00
Paul Fitzpatrick
6b3ac07ca7 (core) process GristDocAPI calls from custom widgets in the client
Summary:
Processing these calls in the client, rather than passing them on
to the backend, means that access rules are more straightforward to
apply.

An unrelated fix is included to filter _grist_ tables when fetched
individually - metadata could leak through this path.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2954
2021-07-30 10:41:32 -04:00
Paul Fitzpatrick
95cc2eb282 (core) read document as owner in pre-fork mode, if have sufficient access to it
Summary:
This tweaks pre-fork mode to make the user's experience a bit more seamless.
Pre-fork mode is where the user has opened a document with intent to
fork it, but actual forking (with allocation of a new document id)
is postponed until they make their first change.

The tweak makes the user an owner for granular access purposes, if
forking is permitted.  So data visible only to owners because of
access rules will be visible to them.  As always, any edits would
go to a separate new copy.

A remaining tricky corner is what to do about "View As" functionality
on forks.  Fork sharing cannot be controlled, so the "Users -> View As"
functionality isn't available.  Perhaps the "Users" button on a fork
could encourage doing a save-copy and inviting users, or offer some
dummy users?  In any case, this diff doesn't change anything with
that corner.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2931
2021-07-21 14:52:31 -04:00
George Gevoian
e5eeb3ec80 (core) Add 'user' variable to trigger formulas
Summary:
The 'user' variable has a similar API to the one from access rules: it
contains properties about a user, such as their full name and email
address, as well as optional, user-defined attributes that are populated
via user attribute tables.

Test Plan: Python unit tests.

Reviewers: alexmojaki, paulfitz, dsagal

Reviewed By: alexmojaki, dsagal

Subscribers: paulfitz, dsagal, alexmojaki

Differential Revision: https://phab.getgrist.com/D2898
2021-07-15 15:18:32 -07:00
Paul Fitzpatrick
6f02987d10 (core) allow undos to be partial, if access control prohibits some part of them
Summary:
This is a somewhat experimental change, that will implement permitted parts of an undo if not all parts are permitted.  This is in preparation for trigger columns, where it may become common for a change in a record resulting in a change to an automatic change to another that the user cannot edit directly.  How to undo such an action is somewhat unclear.  One option is to undo the permitted parts, and then the triggers can rerun.

The general case is a bit of a can of worms, and feels adjacent to merging/rebasing etc.

Oh: it would probably be important in general to communicate to the user that an undo was partial, but this diff doesn't do that.  It would need some new plumbing.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2839
2021-06-10 18:26:11 -04:00
Alex Hall
2f3a0e0c7f (core) Showing censored values as a grey cell
Test Plan: Block read access to column A based on the condition rec.B == 1. Then setting B = 1 in a row makes the cell under A grey.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: paulfitz, dsagal

Differential Revision: https://phab.getgrist.com/D2828
2021-06-07 13:11:41 +02:00
Dmitry S
d1c1416d78 (core) Add rules to eslint to better match our coding conventions.
Summary:
We used tslint earlier, and on switching to eslint, some rules were not
transfered. This moves more rules over, for consistent conventions or helpful
warnings.

- Name private members with a leading underscore.
- Prefer interface over a type alias.
- Use consistent spacing around ':' in type annotations.
- Use consistent spacing around braces of code blocks.
- Use semicolons consistently at the ends of statements.
- Use braces around even one-liner blocks, like conditionals and loops.
- Warn about shadowed variables.

Test Plan: Fixed all new warnings. Should be no behavior changes in code.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2831
2021-05-24 12:56:18 -04:00
Paul Fitzpatrick
15723d1300 (core) check for +S bit early for changes that include formulas
Summary:
Currently, to compute intermediate steps in a bundle, the bundle
is sent to the data engine to process.  Then, if the intermediate
steps break a rule, it is reverted.  One problem introduced by
checking permissions this late is that the data engine can be
exposed for formulas with python code by users who don't have the
right to change formulas.  This diff pre-checks cases that change
formulas.

Test Plan: added a test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2816
2021-05-14 09:18:05 -04:00
Paul Fitzpatrick
3a0ec7b103 (core) be less fussy about rec/newRec distinction for creates/deletions
Summary: For row creations and deletions, treat `rec` and `newRec` variables as identical. This simplifies writing a single rule that controls multiple permissions.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2812
2021-05-13 08:54:49 -04:00
Paul Fitzpatrick
d0d3d3d0c9 (core) discount indirect changes for access control purposes
Summary:
This diff discounts indirect changes for access control purposes.  A UserAction that updates a cell A, which in turn causes changes in other dependent cells, will be considered a change to cell A for access control purposes.

The `engine.apply_user_actions` method now returns a `direct` array, with a boolean for each `stored` action, set to `true` if the action is attributed to the user or `false` if it is attributed to the engine.  `GranularAccess` ignores actions attributed to the engine when checking for edit rights.

Subtleties:
 * Removal of references to a removed row are considered direct changes.
 * Doesn't play well with undos as yet.  An action that indirectly modifies a cell the user doesn't have rights to may succeed, but it will not be reversible.

Test Plan: added tests, updated tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2806
2021-05-12 11:26:21 -04:00
Paul Fitzpatrick
729774552f (core) make sure Calculate action has full access
Summary:
Exceptional document operations (particularly `system` and `nascent`
operations) should never be denied by a granular access rule.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2792
2021-04-29 08:48:07 -04:00
Paul Fitzpatrick
9696e24aac (core) always allow owners to edit access rules, so they don't get stuck
Summary: Access to structural tables currently depends on SchemaEdit permission.  We now make an exception for owner access to _grist_ACLResources and _grist_ACLRules, giving them unconditional access.  It was too easy for owners to lock themselves out of editing access rules.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2790
2021-04-28 10:17:36 -04:00
Dmitry S
526b0ad33e (core) Configure more comprehensive eslint rules for Typescript
Summary:
- Update rules to be more like we've had with tslint
- Switch tsserver plugin to eslint (tsserver makes for a much faster way to lint in editors)
- Apply suggested auto-fixes
- Fix all lint errors and warnings in core/, app/, test/

Test Plan: Some behavior may change subtly (e.g. added missing awaits), relying on existing tests to catch problems.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2785
2021-04-26 18:54:55 -04:00
Paul Fitzpatrick
ee3a8a2b85 (core) streamline mixedColumns case of granular access control
Summary:
This removes some unintentional repetition of work when there are
no row-level rules (there was a missing `return`).

Test Plan: existing tests pass

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2782
2021-04-23 16:14:34 -04:00
Paul Fitzpatrick
2dfa427d63 (core) support subscriptions to a doc with row-dependent column read permissions
Summary: This addresses a weakness in the following case: rules controlling view access for a column, with a dependency on the values of other columns. We had disabled support for such rules, since the existing implementation worked only on table loads and not on broadcast changes. This diff adds in logic to enrich broadcasts as needed, and allows such rules.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2774
2021-04-16 14:42:34 -04:00
Paul Fitzpatrick
35303fad21 (core) disentangle row and metadata steps in granular access calculations
Summary:
When adding robustness to schema changes to granular access control,
a calculation of intermediate row states that was previously done
semi-intelligently on need started happening less intelligently.
This diff separates out the row state calculations from metadata
state calculations so that one can happen without the other.

Test Plan: extended a test.  Also did some manual checks.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2773
2021-04-16 08:28:51 -04:00
Paul Fitzpatrick
d64461cd81 (core) optimization: remove lodash/pullAt
Summary:
For a long array with removals proportional to that length,
lodash/pullAt becomes slow due to doing one splice per removal.
This diff swaps in an alternate implementation that doesn't become
quadratic.  On a 250k-row doc with a row-level access rule, this improves
initial page load for a viewer with access to half the rows from minutes
to seconds.

Test Plan: added test; did manual benchmarking

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2777
2021-04-15 21:18:47 -04:00
Paul Fitzpatrick
9d1bc5a518 (core) make AccessRules and FullCopies effective
Summary:
This allows `*SPECIAL:AccessRules` to give read access to the access rules to more users, and `*SPECIAL:FullCopies` to grant download/copy rights to more users.

This diff also changes forks to be owned by the user who forked them (previously they were an editor), since that feels more natural.

Test Plan: Added and updated tests.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2760
2021-03-25 15:05:26 -04:00
Dmitry S
e14488bcc8 (core) Add support for special ACL rules, for viewing rules and downloading documents.
Summary:
- Use special ACLResources of the form "*SPECIAL:<RuleType>" to represent
  special document-wide rules.
- Include default rules that give Read access to these resources to Owners only.
- Add UI with a checkbox to give access to everyone instead.
- Allow expanding the UI for advanced configuration.

- These rules don't actually have any behavior yet.

Test Plan: WIP

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2764
2021-03-25 10:28:05 -04:00
Paul Fitzpatrick
0c5f7cf0a7 (core) add SELF_HYPERLINK() function for generating links to the current document
Summary:
 * Adds a `SELF_HYPERLINK()` python function, with optional keyword arguments to set a label, the page, and link parameters.
 * Adds a `UUID()` python function, since using python's uuid.uuidv4 hits a problem accessing /dev/urandom in the sandbox.  UUID makes no particular quality claims since it doesn't use an audited implementation.  A difficult to guess code is convenient for some use cases that `SELF_HYPERLINK()` enables.

The canonical URL for a document is mutable, but older versions generally forward.  So for implementation simplicity the document url is passed it on sandbox creation and remains fixed throughout the lifetime of the sandbox.  This could and should be improved in future.

The URL is passed into the sandbox as a `DOC_URL` environment variable.

The code for creating the URL is factored out of `Notifier.ts`. Since the url is a function of the organization as well as the document, some rejiggering is needed to make that information available to DocManager.

On document imports, the new document is registered in the database slightly earlier now, in order to keep the procedure for constructing the URL in different starting conditions more homogeneous.

Test Plan: updated test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2759
2021-03-18 19:37:07 -04:00
Paul Fitzpatrick
c37a04c578 (core) freshen "view as user" behavior
Summary:
Now as the user an owner might choose to view their document as
is likely to not have access to rules, it is better to start
viewing on the default document page rather than /p/acl.

The "Access Rules" link is grayed out when in "view as" mode for
now (improvements are planned).

Test Plan: updated test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2743
2021-03-03 09:40:20 -05:00
Paul Fitzpatrick
937214d927 (core) update column access logic to respect wildcards
Summary:
Previously, if some columns are allowed and the rest are denied,
the client could see unnecessary blank columns.  This diff cleans
up column metadata censorship.  It also adds a small tweak to
retain the `manualSort` column when filtering columns for a mixed
access table.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2742
2021-03-03 09:39:36 -05:00
Paul Fitzpatrick
4ab096d179 (core) granular access control in the presence of schema changes
Summary:
 - Support schema changes in the presence of non-trivial ACL rules.
 - Fix update of `aclFormulaParsed` when updating formulas automatically after schema change.
 - Filter private metadata in broadcasts, not just fetches.  Censorship method is unchanged, just refactored.
 - Allow only owners to change ACL rules.
 - Force reloads if rules are changed.
 - Track rule changes within bundle, for clarity during schema changes - tableId and colId changes create a muddle otherwise.
 - Show or forbid pages dynamically depending on user's access to its sections. Logic unchanged, just no longer requires reload.
 - Fix calculation of pre-existing rows touched by a bundle, in the presence of schema changes.
 - Gray out acl page for non-owners.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2734
2021-03-01 13:49:31 -05:00
Paul Fitzpatrick
6af811f7ab (core) give more detailed reasons for access denied when memos are present
Summary:
With this change, if a comment is added to an ACL formula, then that comment will be offered to the user if access is denied and that rule could potentially have granted access.

The code is factored so that when access is permitted, or when partially visible tables are being filtered, there is little overhead. Comments are gathered only when an explicit denial of access.

Test Plan: added tests, updated tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2730
2021-02-15 17:02:24 -05:00
Dmitry S
7c81cf2368 (core) Change array.reverse() in GranularAccess to avoid accidental in-place reversal.
Test Plan: Added a test case to tickle the bug this was causing.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2718
2021-01-28 15:07:33 -05:00
Dmitry S
6f9b85fc8c (core) Show a clearer message when actions are blocked by ACL rules
Summary:
- This replaces the message "Unexpected Error / Access Denied / Report a problem" with a
  one-line "Blocked by access rules".

Test Plan: Only tested manually

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2712
2021-01-22 10:21:36 -05:00
Paul Fitzpatrick
f4366a01b3 (core) tweak meaning of newRec to be state at end of bundle
Summary:
This redefines `newRec` to be the state at the end of a bundle,
for the purposes of modifying a document.  Updates and adds tests
for creation/updates of rows that are now more intuitive hopefully.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2707
2021-01-21 14:10:53 -05:00
Paul Fitzpatrick
3ad9b18ddf (core) allow a doc owner to test access as a different user
Summary:
This adds back-end support for query parameters `aclAsUser_` and
`aclAsUserId_` which, when either is present, direct Grist to
process granular access control rules from the point of view
of that user (specified by email or id respectively).

Some front end support is added, in the form of a tag that
shows up when in this mode, and a way to cancel the mode.
No friendly way to initiate the mode is offered yet.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2704
2021-01-15 18:45:57 -05:00
Paul Fitzpatrick
438f259687 (core) start reconciling forking with granular access
Summary:
This allows a fork to be made by a user if:
 * That user is an owner of the document being forked, or
 * That user has full read access to the document being forked.

The bulk of the diff is reorganization of how forking is done.  ActiveDoc.fork is now responsible for creating a fork, not just a docId/urlId for the fork. Since fork creation should not be limited to the doc worker hosting the trunk, a helper endpoint is added for placing the fork.

The change required sanitizing worker allocation a bit, and allowed session knowledge to be removed from HostedStorageManager.

Test Plan: Added test; existing tests pass.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2700
2021-01-12 14:08:49 -05:00
Dmitry S
5deac68315 (core) Update ACL resources/rules when tables/columns get renamed
Summary:
- Placed rule-updating functions in acl.py.
- Reset UI when rules update externally, or alert the user to reset if there
  are pending local changes.
- Removed some unused and distracting bits from client-side DocModel.

A few improvements related to poor error handling:
- In case of missing DocActions (tickled by broken ACL rule handling), don't
  add to confusion by attempting to process bad actions
- In case of missing attributes in ACL formulas, return undefined rather than
  fail; the latter creates more problems.
- In case in invalid rules, fail rather than skip; this feels more correct now
  that we have error checking and recovery option, and helps avoid invalid rules.
- Prevent saving invalid rules with an empty ACL formula.
- Fix bug with rule positions.

Test Plan: Added a python and browser test for table/column renames.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2698
2021-01-04 22:03:01 -05:00