Summary:
- Introduce a new SQLiteDB migration, which adds DB columns for formula columns
- Newly added columns have the special ['P'] (pending) value in them
(in order to show the usual "Loading..." on the first load that triggers the migration)
- Calculated values are added to .stored/.undo fields of user actions.
- Various changes made in the sandbox to include .stored/.undo in the right order.
- OnDemand tables ignore stored formula columns, replacing them with special SQL as before
- In particular, converting to OnDemand table leaves stale values in those
columns, we should maybe clean those out.
Some tweaks on the side:
- Allow overriding chai assertion truncateThreshold with CHAI_TRUNCATE_THRESHOLD
- Rebuild python automatically in watch mode
Test Plan: Fixed various tests, updated some fixtures. Many python tests that check actions needed adjustments because actions moved from .stored to .undo. Some checks added to catch situations previously only caught in browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2645
Summary: Adds a granular access clause for columns. Permissions can be specified for a set of columns within a table. Permissions accumulate over clauses, in a way that is intended as a placeholder pending final design.
Test Plan: Added tests. Tested manually that updates to private columns are not sent to people who don't have access to them. There are a lot of extra tests needed and TODOs to be paid down after this experimental phase.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2651
Summary:
Deliberate changes:
* save snapshots to s3 prior to migrations.
* label migration snapshots in s3 metadata.
* avoid pruning migration snapshots for a month.
Opportunistic changes:
* Associate document timezone with snapshots, so pruning can respect timezones.
* Associate actionHash/Num with snapshots.
* Record time of last change in snapshots (rather than just s3 upload time, which could be a while later).
This ended up being a biggish change, because there was nowhere ideal to put tags (list of possibilities in diff).
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2646
Summary:
This is a prototype for expanding the conditions that can be used in granular ACLs.
When processing ACLs, the following variables (called "characteristics") are now available in conditions:
* UserID
* Email
* Name
* Access (owners, editors, viewers)
The set of variables can be expanded by adding a "characteristic" clause. This is a clause which specifies:
* A tableId
* The name of an existing characteristic
* A colId
The effect of the clause is to expand the available characteristics with all the columns in the table, with values taken from the record where there is a match between the specified characteristic and the specified column.
Existing clauses are generalized somewhat to demonstrate and test the use these variables. That isn't the main point of this diff though, and I propose to leave generalizing+systematizing those clauses for a future diff.
Issues I'm not dealing with here:
* How clauses combine. (The scope on GranularAccessRowClause is a hack to save me worrying about that yet).
* The full set of matching methods we'll allow.
* Refreshing row access in clients when the tables mentioned in characteristic tables change.
* Full CRUD permission control.
* Default rules (part of combination).
* Reporting errors in access rules.
That said, with this diff it is possible to e.g. assign a City to editors by their email address or name, and have only rows for those Cities be visible in their client. Ability to modify those rows, and remain updates about them, remains under incomplete control.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2642
Summary:
This implements a form of row-level access control where for a
given table, you may specify that only owners have access to
rows for which a given column has falsy values.
For simplicity:
* Only owners may edit that table.
* Non-owners with the document open will have forced
reloads whenever the table is modified.
Baby steps...
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2633
Summary:
Add a unittest that start the server with limited memory, and build just enough
of ActionHistory to crash the server before this fix, and not after.
Test Plan: Tested manually with various memory prints, and added a test.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2616
Summary:
* Extends `/api/docs/docId1/compare/docId2` endpoint with a `detail=1` option to include details of what changed in the document content.
* Adds an `/api/docs/docId/compare?left=HASH&right=HASH` endpoint for comparing two versions of a single document. This is needed to implement the extension to `/api/docs/docId1/compare/docId2`.
* Adds a `HashUtil` class to allow hash aliases like `HEAD` and `HEAD~`.
Everything is a bit crude:
* Changes are expressed as ActionSummary objects, which aren't fully fleshed out.
* Extra data about formula columns is inserted in an inflexible way.
This is extracted and cleaned up from https://phab.getgrist.com/D2600.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2614
Summary:
This is an incremental step in granular access control. Using
a temporary `{colIds: '~o structure'}` representation in the
`_grist_ACLResources` table, the document structure can be set
to be controlled by owners only.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2613
Summary:
This makes it possible to serve a table or tables only to owners.
* The _grist_ACLResources table is abused (temporarily) such that rows of the form `{colId: '~o', tableId}` are interpreted as meaning that `tableId` is private to owners.
* Many websocket and api endpoints are updated to preserve the privacy of these tables.
* In a document where some tables are private, a lot of capabilities are turned off for non-owners to avoid leaking info indirectly.
* The client is tweaked minimally, to show '-' where a page with some private material would otherwise go.
No attempt is made to protect data from private tables pulled into non-private tables via formulas.
There are some known leaks remaining:
* Changes to the schema of private tables are still broadcast to all clients (fixable).
* Non-owner may be able to access snapshots or make forks or use other corners of API (fixable).
* Changing name of table makes it public, since tableId in ACLResource is not updated (fixable).
Security will require some work, the attack surface is large.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2604
Summary: This makes the user's role (owner/editor/viewer) available in ActiveDoc methods. No use of that information is made yet, other than to log it. The bulk of the diff is getting a handle on the various ways the methods can be called, and systematizing it a bit more. In passing, access control is added to broadcasts of document changes, so users who no longer have access to a document do not receive changes if they still have the document open.
Test Plan: existing tests pass; test for broadcast access control added
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2599
Summary: This moves enough server material into core to run a home server. The data engine is not yet incorporated (though in manual testing it works when ported).
Test Plan: existing tests pass
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2552
