Summary:
* adds a smoke test to grist-core
* fixes a problem with highlight.js failing to load correctly
* skips survey for default user
* freshens docker build
Utility files in test/nbrowser are moved to core/test/nbrowser, so that gristUtils are available there. This increased the apparent size of the diff as "./" import paths needed replacing with "test/nbrowser/" paths. The utility files are untouched, except for the code to start a server - it now has a small grist-core specific conditional in it.
Test Plan: adds test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2768
Summary:
- Allows ColumnFilter to add/delete keys by batch
- Add options limitShown to ColumnFilterMenu
- Add summary checkboxes Other Matching/Other Non-Matching/Other Values
- Adds missing type to chai declaration
Test Plan:
- Adds project test to new file projects/ColumnFilterMenu2
- Adds nbrowser test to new file nbrowser/ColumnFilterMenu
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2763
Summary:
- When Public Edit access is enabled, Access Rules page shows a warning and
prevents saving rules.
- When any ACL rules are present, attempts to set Public access to Editor role
get downgraded to Viewer role, with a warning notification.
- No checks are made on the server side, so the combination may be achieved via
the API (but we may block it in the future).
Test Plan: Added a test case.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2767
Summary:
- Remove support for aclUI=1 URL parameter, making it the default.
- Add 'BETA' tag to the Access Rules link in side panel.
- Remove all mentions of aclUI=1 in tests.
Test Plan: Updated tests should pass
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2766
Summary:
This allows `*SPECIAL:AccessRules` to give read access to the access rules to more users, and `*SPECIAL:FullCopies` to grant download/copy rights to more users.
This diff also changes forks to be owned by the user who forked them (previously they were an editor), since that feels more natural.
Test Plan: Added and updated tests.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2760
Summary:
- Use special ACLResources of the form "*SPECIAL:<RuleType>" to represent
special document-wide rules.
- Include default rules that give Read access to these resources to Owners only.
- Add UI with a checkbox to give access to everyone instead.
- Allow expanding the UI for advanced configuration.
- These rules don't actually have any behavior yet.
Test Plan: WIP
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2764
Summary:
Previously in {{D1053}} we switched to using BLOB as the "type" for all columns, to prevent SQLite from casting data unexpectedly. This diff now returns to more meaningful types. We apply marshalling to values when being placed in a column where a cast might occur, to inhibit such casting.
The benefit is that Grist documents become easier to interact with via regular database clients/libraries, which often rely on the column type more than a purely SQLite tool would.
On column type conversion, we run all blobs in the column through a decode/encode cycle so if they no longer need to be marshalled they revert to native type. This could be optimized further, it is somewhat brute force.
Test Plan: Updated tests and reference document
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2755
Summary:
- Simply apply search against word label instead of the key
- Fix a tricky bug of observable dependencies that appeared while working on this diff.
Test Plan: Added a browser test for searching date in the filter menu.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2762
Summary: this adds constants for user access roles, to facilitate autocomplete.
Test Plan: updated tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2761
Summary:
* Adds a `SELF_HYPERLINK()` python function, with optional keyword arguments to set a label, the page, and link parameters.
* Adds a `UUID()` python function, since using python's uuid.uuidv4 hits a problem accessing /dev/urandom in the sandbox. UUID makes no particular quality claims since it doesn't use an audited implementation. A difficult to guess code is convenient for some use cases that `SELF_HYPERLINK()` enables.
The canonical URL for a document is mutable, but older versions generally forward. So for implementation simplicity the document url is passed it on sandbox creation and remains fixed throughout the lifetime of the sandbox. This could and should be improved in future.
The URL is passed into the sandbox as a `DOC_URL` environment variable.
The code for creating the URL is factored out of `Notifier.ts`. Since the url is a function of the organization as well as the document, some rejiggering is needed to make that information available to DocManager.
On document imports, the new document is registered in the database slightly earlier now, in order to keep the procedure for constructing the URL in different starting conditions more homogeneous.
Test Plan: updated test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2759
Summary:
- Update styling of label, id, and "derived ID from label" checkbox.
- Implement a label which shows 'Data Column' vs 'Formula Column' vs 'Empty Column',
and a dropdown with column actions (such as Clear/Convert)
- Implement new formula display in the side-panel, and open the standard
FormulaEditor when clicked.
- Remove old FieldConfigTab, of which now very little would be used.
- Fix up remaining code that relied on it (RefSelect)
Test Plan: Fixed old tests, added new browser cases, and a case for a new helper function.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2757
Summary:
- Makes search input alway visible
- Gives search input focus on open
- Adds `Future Values` Checkbox
- Show `All Shown` `All Excpet` when values are filtered
- Show `Others` instead of `Future Values` when values are filtered
- Escape close search input
- Enter does the same as `All Shown` when filtering values
Test Plan:
- Updated existing projects and nbrowser test
- Adds new projects test
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2753
Summary:
- Switch code that compares user strings to use localeCompare() based on Intl.Collator.
- Use en-US locale for now. (Ideally should be a document property.)
- Note that with this change, sorting is also becoming case-insensitive (which
seems an improvement)
- Updated a sorted test fixture
- Updated a browser test with lots of unicode to expect different order.
- Added a bit of unicode to test ordering in Reference autocomplete dropdown.
Test Plan: Fixed / updated tests
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2758
Summary:
a script I used when preparing access rule documentation.
Committing it since the feature in still in beta, so keeping the
script around may save some time, even if it rusts quickly.
I didn't invest time in cleaning it up.
Test Plan: manual
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2756
Summary:
This particular combination of features is not built out - data will be
censored but changes to data will not. So the user will now get an error
if they try to do it. Existing rules of this kind will continue to
operate as before, and can be set via the api.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2751
Summary:
- When in View-As mode, clicking the Access Rules page now shows a tooltip
with a link to return to normal mode and open the Access Rules page.
- A "revert" button is shown next to the item with the same behavior.
- Implemented hoverTooltip() with various options. (It will have other uses.)
- Simplify creation of links based on UrlState:
- Allow merging with previous urlState using a function
- Add a helper function to merge in aclAsUser parameter.
- Add setHref() method to UrlState
Test Plan:
Added test cases:
- for tooltips generally in test/projects
- for updating UrlState using a callback
- for Access Rules tooltip and button behavior
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2749
Summary:
- reuses the textInput form the editableLabel module
- adds a isValidHex utility function to gutil
Test Plan: - Adds test to the project test
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2744
Summary: This fixes a bug where one client's access control limits could remove data from others via a cache.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2748
Summary:
- No longer convert data columns to formula by typing a leading "=". Instead,
show a tooltip with a link to click if the conversion was intended.
- No longer convert a formula column to data by deleting its formula. Leave the
column empty instead.
- Offer the option "Convert formula to data" in column menu for formulas.
- Offer the option to "Clear column"
- If a subset of rows is shown, offer "Clear values" and "Clear entire column".
- Add logic to detect when a view shows a subset of all rows.
- Factor out showTooltip() from showTransientTooltip().
- Add a bunch of test cases to cover various combinations (there are small
variations in options depending on whether all rows are shown, on whether
multiple columns are selected, and whether columns include data columns).
Test Plan: Added a bunch of test cases.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2746
Test Plan: Added a test case that fails without the fix.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2747
Summary:
Previously, it was forbidden to have two rule sets with overlapping columns,
since that could introduce an dependency on order of evaluation without
the user having a way to control that order. This diff permits such rule sets
if the are compatible in a very simple way -- all allows or all denies.
Anything more complicated (even if actually order independent) remains forbidden.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2745
Summary:
Now as the user an owner might choose to view their document as
is likely to not have access to rules, it is better to start
viewing on the default document page rather than /p/acl.
The "Access Rules" link is grayed out when in "view as" mode for
now (improvements are planned).
Test Plan: updated test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2743
Summary:
Previously, if some columns are allowed and the rest are denied,
the client could see unnecessary blank columns. This diff cleans
up column metadata censorship. It also adds a small tweak to
retain the `manualSort` column when filtering columns for a mixed
access table.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2742
Summary:
- Fix transparency support on color select
- Fix z-index conflicts with color select and right panel
- Makes widget's default text color visible to color select
Test Plan: - Updates nbrowser/CellColor and browser/Widget.test to support new interface. Should not cause regression.
Reviewers: paulfitz, dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2735
Summary:
- Support schema changes in the presence of non-trivial ACL rules.
- Fix update of `aclFormulaParsed` when updating formulas automatically after schema change.
- Filter private metadata in broadcasts, not just fetches. Censorship method is unchanged, just refactored.
- Allow only owners to change ACL rules.
- Force reloads if rules are changed.
- Track rule changes within bundle, for clarity during schema changes - tableId and colId changes create a muddle otherwise.
- Show or forbid pages dynamically depending on user's access to its sections. Logic unchanged, just no longer requires reload.
- Fix calculation of pre-existing rows touched by a bundle, in the presence of schema changes.
- Gray out acl page for non-owners.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2734
Summary:
- Adjust ExampleCard css to fit on x-small screens, including its close button,
so that it may be closed.
- In TopBar on small screens, hide Notifications button and default action for
Share; adjust spacing.
- Add overflow: hidden to PagePanels, to make sure it doesn't slide off of
visible area.
Test Plan: Tested on iOS.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2741
Summary:
- Make unsupported browser warning into an unobtrusive one-liner, similar in
style to notifications.
- Move browser warning details into a support page, linked from "Learn more" link.
- Show different mobile and desktop warnings.
- Once dismissed, remember dismissal for a year rather than just for the session.
- Turn the Sign-In button (for anon users) into a menu (for the sake of exposing
the Toggle Mobile Mode option)
- Improve styling of HomeIntro screens when on small screen.
- Flip the default for setting mobile viewport to true
Test Plan: Added minor unittest for localStorageBoolObs; fixed other affected tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2738
Summary:
- The error appeared recently, due to more frequent resize calls (added for mobile)
- In fact, charts' own resize logic can now be simplified.
Test Plan: Added a test case (which fails without the fix)
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2739
Summary:
Previous diff (https://phab.getgrist.com/D2736) broke a test, which I assumed
was unrelated, but should have checked.
In fact, a few tests were changed to pass with the changed default color
(although no such change was planned). This diff reverts those changes, and
reverts also the change to error cells, so that their text is also black again.
Test Plan: Checked affected tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2737
Summary:
Diff https://phab.getgrist.com/D2720 inadvertently changed the default color of
text in cells from black to #333 (inherited from bootstrap's default for
<body>). This change reverts the default back to black.
Test Plan: Checked manually.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2736
Summary:
Component is implemented as a grainjs ui component and can be tested using `yarn serve-projects`.
This diff does not bring color select to Grist just yet.
Follow up:
- Make it possible to set a custom color by typing hex value directly in.
- Disable the button while save call is pending.
Test Plan:
- Adds a project test
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2733
Summary:
With this change, if a comment is added to an ACL formula, then that comment will be offered to the user if access is denied and that rule could potentially have granted access.
The code is factored so that when access is permitted, or when partially visible tables are being filtered, there is little overhead. Comments are gathered only when an explicit denial of access.
Test Plan: added tests, updated tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2730
Summary:
This updates the logic for skipping unchanged rows to take removed
rows into account.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2732
Summary:
CSS differences for small-screen may get applied when printing. Most such
differences are for elements that get hidden for printing anyway. For
collapsing view sections, it's important not to apply small-screen CSS for
printing.
Test Plan: Tested manually on Chrome and Firefox.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2731
Test Plan: Tested manually on iPhone and android simulator. On Desktop mode, resize isn't needed, but seems harmless (I can't see any artifacts caused by it).
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2728
Summary:
Column conversions involve changes to metadata tables such as
_grist_Tables_column. When fetched (from GranularAccess), ExpandedQuery used to
fail with 'table not found' because there is no metadata for metadata tables.
This diff limits the need for metadata in ExpandedQuery to when it's actually
needed (to implmement some formulas for on-demand tables), which no longer
interferes with GranularAccess.
Test Plan: Added a test case that reproduces the issue before the fix.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2729
Summary:
This solves the issue of double-tapping cells when NOT in mobile mode, which is
caused by delayed simulation of click events. (In mobile mode, modern browsers
don't add a delay either way.)
Also avoid unintended opening of the editor by checking that the double-tap is
on the same cell.
Test Plan: Tested manually on iPhone and Chrome emulator.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2727
Summary:
Implement an approach to makind multi-section screens usable on mobile by
collapsing inactive sections to a small area. When clicked, they become active
and expand, while the rest of the sections are collapsed.
Test Plan: Added a basic test case of collapsing inactive sections.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2725
Summary:
. Makes cell color work well in comparison mode
. Do not apply cell color to the add new row
. Allow to change color for all widget (including changing color for the checkbox and the switch widget)
. Fix an issue that was setting color to black when opening the picker
. Do not apply color to invalid cell
Test Plan: . Added nbrowser/CellColor
Reviewers: paulfitz, dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2720
Summary:
- Add isNarrowScreenObs() observable.
- Remove optimizeNarrowScreen flag (now assumed always true).
- Added viewport support and mobile tweaks to Error/Billing/Welcome pages.
- Fix responsiveness of panel transitions, and of side panel state.
- Close left panel on navigation to another page or workspace.
- Start panels collapsed in both doc and docmenu cases.
Test Plan: Tested manually, and fixed tests to accept the new behavior.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2726
Summary:
In the transition from a "pre-fork" to fork, when embedded,
the fork ends up being read-only and changes fail. This commit
avoids applying the read-only default to forks.
If a user actually wants to specifically embed a fork as read-only,
they can still do so, by explicitly adding `/m/view`.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2723
Test Plan: Added a check for bottom-footer on small screen to the InterfaceStyle test.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2722
Summary:
- Add custom handling for dblclick on mobile, to allow focusing editor.
- In place of Clipboard.js, use a FocusLayer with document.body as the default focus element.
- Set maximum-scale on iOS viewport to prevent auto-zoom.
- Reposition the editor on window resize when editing a cell, which is a normal
occurrence on Android when virtual keyboard is shown.
- Add Save/Cancel icon-buttons next to cell editor on mobile.
Test Plan: Tested manually on Safari / FF on iPhone, and on Chrome on Android emulator.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2721
Test Plan: Added a test case to tickle the bug this was causing.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2718
Summary:
The way linkId was set on actions to tie them together for undo bundling was
incorrect. This diff fixes it by moves the setting of linkIds to Sharing.ts,
which already serializes the processing of actions.
Test Plan: Added a test case for submitting actions together while bundling (which fails without this change).
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2716
Summary:
Currently, an undo of a non-last action can leave the doc in an inconsistent
state. For example, it may remove a table, but fail to remove all columns of
it from metadata. We normally check that schema corresponds to metadata, but
stray columns were not visible to this check, and instead caused later table
additions to fail.
This diff fixes the check to fail the action that causes stray columns, and
to restore the doc to a consistent state.
Note that this only handles schema-metadata inconsistencies, but an undo of a
non-last action can easily create other surprises.
Test Plan: Added a test case that triggered inconsistency before, and now triggers a failed undo.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2715
Summary:
Side panels sliding out of the viewport was causing the
browser window to be scrollable, hence it was possible to scroll the
page panels out of the viewport. Solution is to use fixed positioning
instead of absolute.
Test Plan: Tested manually on FF and Chrome.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2714
Summary:
Adds POST /api/housekeeping/docs/:docId/force-reload, which allows the support user to force a document to reload (even if they don't otherwise have access to the document).
This could be done without a separate endpoint, but that turned out a bit messy, and there's some advantage to quarantining the exceptional authorization somewhere it could be easily elaborated.
Test Plan: adds test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2713
Summary:
Diff makes side panels slide horizontally when opening/closing them.
Caveats:
. Right panel: even though the panel do transition nicely, the content however disappears suddently. We could prevent disposal of the dom by removing the two below lines. But it's hard to tell what possible side effect we could get from it as I don't know why these line were added in the first place. I could investigate further, but maybe it's already good enough as it is.
```
private _buildContentDom() {
return dom.domComputed((use) => {
// if (!use(this._isOpen)) { return null; } // remove line
const tool = use(this._extraTool);
```
```
private _buildHeaderDom() {
return dom.domComputed((use) => {
// if (!use(this._isOpen)) { return null; } // remove line
const tool = use(this._extraTool);
return tool ? this._buildToolHeader(tool) : this._buildStandardHeader();
```
Test Plan: Tested manually on desktop environnment with FF and chrome by shrinking the window.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2710
Summary:
- This replaces the message "Unexpected Error / Access Denied / Report a problem" with a
one-line "Blocked by access rules".
Test Plan: Only tested manually
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2712
Summary: The list of users allows copying users' emails to clipboard, and viewing the doc as that user.
Test Plan: Added a basic test case
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2711
Summary:
- Enable narrow-screen layout for home page
- Clean up margins/spacing on small-screen home page
- Use "<768" as small-screen condition rather than "<=768".
- Include meta-viewport tag conditionally, off by default.
- Include "Toggle Mobile Mode" option in AccountMenu to toggle it on.
- In a test, add an after() clause to restore window size even when test fails
Test Plan: Only tested manually on iPhone (Safari & FF).
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: cyprien
Differential Revision: https://phab.getgrist.com/D2708
Summary:
This redefines `newRec` to be the state at the end of a bundle,
for the purposes of modifying a document. Updates and adds tests
for creation/updates of rows that are now more intuitive hopefully.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2707
Summary:
- closes side bars when tapping content area
- opens left panel when tapping the page name
Test Plan: Includes browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2705
Summary:
This adds back-end support for query parameters `aclAsUser_` and
`aclAsUserId_` which, when either is present, direct Grist to
process granular access control rules from the point of view
of that user (specified by email or id respectively).
Some front end support is added, in the form of a tag that
shows up when in this mode, and a way to cancel the mode.
No friendly way to initiate the mode is offered yet.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2704
Summary:
The goal is that those who can edit ACL rules can create or change rules for
any resource, even if the rules block their own ability to see the resource.
Test Plan: Added a browser test, and a server test for who can call the new method.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2703
Test Plan: Tested Manually, behave well also for long page name.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2702
Summary:
This commit adds a bottom bar with 2 openers button for each side panels.
It takes care of showing/hiding other openers when appropriate.
It also implements the new style for narrow screen with the side panels overlapping the main pane.
Remaining tasks:
- adjust content of the breadcrumbs for narrow screen.
- content of the bottom bar
Test Plan: Adds test to projects/PagePanels
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2701
Summary:
This allows a fork to be made by a user if:
* That user is an owner of the document being forked, or
* That user has full read access to the document being forked.
The bulk of the diff is reorganization of how forking is done. ActiveDoc.fork is now responsible for creating a fork, not just a docId/urlId for the fork. Since fork creation should not be limited to the doc worker hosting the trunk, a helper endpoint is added for placing the fork.
The change required sanitizing worker allocation a bit, and allowed session knowledge to be removed from HostedStorageManager.
Test Plan: Added test; existing tests pass.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2700
Summary:
This adds endpoints that allow the support user to remove unlisted
snapshots for a document, and to remove all action history for
a document.
This does increase what the support user can do, but not in a way
that would be particularly valuable to attack. It would have some
destructive value, for removing history (removing unlisted
snapshots doesn't impact the user, by contrast).
This would simplify some maintenance operations.
Test Plan: added test for snapshots; tested states manually
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2699
Summary:
- Placed rule-updating functions in acl.py.
- Reset UI when rules update externally, or alert the user to reset if there
are pending local changes.
- Removed some unused and distracting bits from client-side DocModel.
A few improvements related to poor error handling:
- In case of missing DocActions (tickled by broken ACL rule handling), don't
add to confusion by attempting to process bad actions
- In case of missing attributes in ACL formulas, return undefined rather than
fail; the latter creates more problems.
- In case in invalid rules, fail rather than skip; this feels more correct now
that we have error checking and recovery option, and helps avoid invalid rules.
- Prevent saving invalid rules with an empty ACL formula.
- Fix bug with rule positions.
Test Plan: Added a python and browser test for table/column renames.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2698
Summary:
- Add ACLColumnList widget for a list of column IDs.
- Replace autocomplete widgets with simpler dropdowns.
- Add select dropdown for the Attribute of UserAttribute rules.
- Switch formula to use ACE editor.
- Factor out customized completion logic from AceEditor.js into a separate file.
- Implement completions for ACL formulas.
- Collect ACL UI files in app/client/aclui
Test Plan: Updated test case, some behavior (like formula autocomplete) only tested manually.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2697
Summary:
- Add headers to tables.
- Change styles to reduce boxes-within-boxes.
- Add validation of table and column IDs, both in UI and on server when saving rules.
- Add autocomplete for tables/columns used for UserAttribute rules.
- Add a fancy widget to set permission bits.
Test Plan: Updated browser test for new UI, added a test case for user attributes.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2695
Summary:
Currently, if a document is created by importing a file, inventory
creation is a little haphazard - it works, but triggers a
"surprise" message. This diff makes initialization of inventory
explicit, so that surprise messages shouldn't happen during
document creation.
Test Plan: manual
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2696
Summary:
This adds a snapshots/remove and states/remove endpoint, primarily
for maintenance work rather than for the end user. If some secret
gets into document history, it is useful to be able to purge it
in an orderly way.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2694
Summary:
After switch to using an inventory file, old document versions were
not in fact being pruned. This corrects that and adds a test
that fails with the previous implementation.
The pruner was operating correctly, but was being applied to an
inventory list rather than s3 directly - and the inventory list
did not pass through version removals to s3.
This fix will leave a stock of undeleted versions that can
be eliminated by an external script (there are alternatives
but that seems simplest overall).
Test Plan: updated test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2693
Summary:
The bug manifested when multiple grid cells were selected, and then user
selected a record in another section that caused the grid to show a different
set of rows. Paste would then go into multiple rows even when they are not
visibly selected.
The test includes some porting of hacks from old browser tests to support
copy-pasting. In particular gu.sendKeys() is a useful alternative to
driver.find('body').sendKeys() which we've been using to work around
driver.sendKeys() limitations, but which apparently causes flakiness with
focus.
Test Plan: Browser test reproduces the bug before the fix.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2692
Summary:
- When adding records, negative rowIds may now be specified. They'll be replaced by proper IDs.
- If these negative IDs are used in Reference columns in subsequent actions in
the same bundle of UserActions, they get replaced with the proper rowIds.
- Use this to sync ACLResources and ACLRules from UI in a single batch of UserActions.
- Resolve the TODOs in GranularAccess test, to no longer need to guess resource rowIds.
Test Plan: Added a python unittest for mapping IDs; updated browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2691
Summary:
The client relies on metadata tables for laying out pages and sections.
These tables are filtered according to what tables the user has access
to, in a crude way. This diff updates the logic to at least support
the table wildcard.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2690
Summary:
- Fix error-handling in bundleActions(), and wait for the full bundle to complete.
(The omissions here were making it impossibly to react to errors from inside bundleActions())
- Catch problematic rules early enough to undo them, by trying out ruleCollection.update()
on updated rules before the updates are applied.
- Added checkAclFormula() call to DocComm that checks parsing and compiling
formula, and reports errors.
- In UI, prevent saving if any aclFormulas are invalid, or while waiting for the to get checked.
- Also fixed some lint errors
Test Plan: Added a test case of error reporting in ACL formulas.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2689
Summary:
Adds an "enter safe mode" option and explanation in modal that appears when a document fails to load, if user is owner. If "enter safe mode" is selected, document is reloaded on server in a special mode. Currently, the only difference is that if the acl rules fail to load, they are replaced with a fallback that grants full access to owners and no access to anyone else. An extra tag is shown to mark the document as safe mode, with an "x" for cancelling safe mode.
There are other ways a document could fail to load than just acl rules, so this is just a start.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2686
Summary:
Disable the submit button while a form submission is pending.
(We don't move to the next page without waiting since we get the page to
redirect to from the server.)
Test Plan: Tested manually by pausing the server and trying to submit multiple times.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2688
Summary:
This changes how user attributes are loaded. They are now loaded
directly from sqlite, with per-session caching. Optimizations
considered but not addressed yet are (1) adding indexes to user attribute
tables and (2) swapping in a thinner sqlite wrapper.
The main benefit of this diff is that changes to user attribute
tables now work. Clients whose user attributes are not changed
see no effect; clients whose user attributes have changed have
their document reloaded.
For the purposes of testing, the diff includes a tweak to
GristWSConnection to be "sticky" to a specific user when reloading
(and support machinery on the server side to honor that). Until
now, if a GristWSConnection reloads, it uses whatever the current
default user is in the cookie-based session, which can change.
This was complicating a test where multiple users were accessing
the same document via different clients with occasional document
reloads.
Code for updating when schema or rule changes happen is moved
around but not improved in any meaningful way in this diff.
Test Plan: existing tests pass; extended test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2685
Summary:
Also added a Chrome option to tests to suppress popup to save passwords,
and unified setting of Chrome options in one shared file.
Test Plan: Tested locally, reproduced the problem by adding a delay in BaseAPI.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2687
Summary:
Some slight difference appeared, possibly during rebase, about including
compare parameters in URLs.
Test Plan: Browser test was failing, now passes
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2683
Summary:
- When viewing a snapshot, list all snapshots for a doc, highlighting the current one.
- Include links in per-snapshot menu to compare-to-current and compare-to-previous.
- Compare links include "beta" tags.
- Set order of comparison to have older on the left, and newer on the right.
Test Plan: Moved out DocHistory test from Snapshots, and added some test cases.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2681
Summary:
This adds any parameters in a document url whose key ends in '_'
into a `user.Link` object available in access control formulas
and in setting up characteristic tables.
This allows, for example, sending links to a document that contain
a hard-to-guess token, and having that link grant access to a
controlled part of the document (invoices for a specific customer
for example).
A `user.Origin` field is also added, set during rest api calls,
but is only tested manually at this point. It could be elaborated
for embedding use-cases.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2680
Summary:
This improves support for access control on document modifications. It adds:
* Checking of create/remove/update access for row-level changes.
* Use of `newRec` variable in formulas.
It is now possible to have distinct clients with read+write access to different rows of the same table.
This is another incremental step. There are deficiencies in actions that include schema changes, and many other lacunae. But the overall flow is taking shape.
Access control is done at the DocAction level, requiring the sandbox to process the UserActions, and then be reverted if the action proves unlawful. This could be optimized away in many simple and important cases, but I'm not sure it is possible to avoid in general.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2677
Summary:
- Factored out ACLRuleCollection into its own file, and use for building UI.
- Moved AccessRules out of UserManager to a page linked from left panel.
- Changed default RulePart to be the last part of a rule for simpler code.
- Implemented much of the UI for adding/deleting rules.
- For now, editing the ACLFormula and Permissions is done using text inputs.
- Implemented saving rules by syncing a bundle of them.
- Fixed DocData to clean up action bundle in case of an early error.
Test Plan: WIP planning to add some new browser tests for the UI
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2678
Summary:
when a document is opened, and Calculate results in
a change, that change is not attributed to 'grist' rather than
to the user's email.
Some minor tweaks included to freshen some related dev scripts.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2679
Summary:
- Replace unicode strings with byte strings when decoding values in sandbox.
- Columns that rely on float values should derive from NumericColumn, so
that set() ensures that a float is stored even if loading an int.
- Parse unmarshallable values (['U']) into an object that can be encoded
back to the same value (rather than info a RaisedException).
- Compare NaN's as equal for deciding whether a change is a no-op.
Unrelated:
- Removed a tiny bit of unhelpful logging
Test Plan:
Added a test case that reproduces several causes of Calculate
discrepancies by loading various values into various types of formula columns.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2676
Summary:
This implements row-level access control for outgoing messages, replacing the document reloading placeholder that was there before.
* Prior to broadcasting messages, GranularAccess is notified of actions+undo.
* While broadcasting messages to different sessions, if we find we need row level access control information, rows before and after the change are reconstructed.
* Messages are rewritten if rows that were previously forbidden are now allowed, and vice versa.
The diff is somewhat under-tested and under-optimized. Next step would be to implement row-level access control for incoming actions, which may result in some rejiggering of the code from this diff to avoid duplication of effort under some conditions.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2670
Summary:
getOrgWorkspaces and getWorkspaces had an unintended feature where
if a user had access to a workspace, they could list all publically
shared documents within that workspace. This diff stops considering
resources shared with everyone@ when listing orgs or workspaces.
Resources shared with anon@ remain listed - this is how the example
workspace operates.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2671
Test Plan: Added a check to the test case verifying the copied link.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2673
Summary:
No behavior changes.
Diff includes an intermediate commit with only renames, for easier review.
Test Plan: Existing tests should pass.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2669
Summary:
- Remove modules related to old login / profile that we don't plan to bring back.
- Remove old unused DocListModel.
- Remove ext* tests that have been skipped and don't work.
- Remove old ModalDialog, and switch its one remaining usage to the newer way.
Test Plan: All tests should pass, and as many as before.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2668
Summary:
- New UI for the modal look mostly following the design prepared previously.
- Use <object> for rendering PDFs (on a Mac works on Firefox, Chrome, Safari; needs checking on Windows)
- While we are at it, use <video> and <audio> for relevant files (object would
render them too, but without the option to disable autoplay).
- Use <object> for unknown types, except for text/html (unsafe) and other text
types (need more work to render well).
- Fix skipping save on Escape or when attachments are unsaved (previously a
noop action was emitted, creating surprises with undo).
- Display extension for files without preview, both in-cell and in the modal.
- Replace tiny "eye" icon to preview particular attachment with double-clicking.
- As an accidental feature, a particular attachment can be previewed by typing 1, 2, 3, etc into cell.
- Renamed PreviewsWidget/PreviewModel to AttachmentsWidget/AttachmentsEditor.
Test Plan: Unified old and new tests for attachments, added new test cases.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2667
Summary:
It can be hard to find changes, even when highlighted, in a table with many rows. This diff replaces long sequences of unchanged rows with a row containing "..."s.
With daff, I found that it is important to do this for sequences of unchanged columns also, but not tackling that yet.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2666
Summary:
- Added fields to _grist_ACLRules for the new Granular ACL representation
- Include a corresponding migration.
- Added ACLPermissions module with merging PermissionSets and converting to/from string.
- Implemented parsing of ACL formulas and compiling them into JS functions.
- Add automatic parsing of ACL formulas when ACLRules are added or updated.
- Convert GranularAccess to load and interpret new-style rules.
- Convert ACL UI to load and save new-style rules.
For now, no attempt to do anything better on the server or UI side, only to
reproduce previous behavior.
Test Plan: Added unittests for new files; fixed those for existing files.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2664
Summary:
This makes data diff rendering robust to changes in the names of tables.
It does not yet show information about those changes, but at least it
won't fail to show table content changes.
Added a missing case to ActionSummary concatenation that came up in
testing.
Test Plan: added test, updated test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2661
Summary:
The new plans for granular access control are different and handled by
node.js. Some of the same tables will be reused, of which we never made
real use before except for expecting certain specific initial records.
This diff removes the old logic, replacing it with a stub that satisfies
the interface expected by other code.
It also removes several unused UserActions: AddUser/RemoveUser/
AddInstance/RemoveInstance.
Test Plan: Existing tests should pass.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2662
Summary:
Loading all user data to run a migration is risky (creates more than usual
memory pressure), and almost never needed (only one migration requires it).
This diff attempts to run migrations using only metadata (_grist_* tables),
but retries if the sandbox tells it that all data is needed.
The intent is for new migrations to avoid needing all data.
Test Plan: Added a somewhat contrived unittest.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2659
Summary:
This makes a small tweak to show cell and row changes in card views
and card list views, and adds a test for it.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2660
Summary:
With recent changes to action history, we can now remove the temporary
`finalRowContent` field from change details, since all the information
we need is now in the ActionSummary.
We also now have more information about the state of the common ancestor,
which previously we could not get either from ActionSummary or from
`finalRowContent`. We take advantage of that to flesh out rendering
differences where there are some changes locally and some changes
remotely.
There's still a lot more to do, this is just one step.
I have added a link to the UI for viewing the comparison. I wouldn't
want to advertise that link until diffs are robust to name changes.
Test Plan: added test, updated tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2658
Summary:
Type conversions and formula tranforms wait for the user and bundle multiple
actions. When an unrelated action is done (e.g. adding a page widget or a
column), we want to finalize the transform before applying it.
The approach turns out fairly complicated. There is an implicit queue of
bundles (which we don't let grow beyond 2, as that's too abnormal). Bundles may
be finalized by a user clicking something, or by an unrelated action/bundle, or
(as before) by transform DOM getting disposed.
- Updated RecordLayout to use bundleActions() helper
- Added support for nesting bundleActions inside another bundle (needed for
setting visibleCol during type change)
- In an unrelated tweak, when in debug-log in ActiveDoc, use a short representation of result.
Test Plan: Added a unittest for action bundling during type transform
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2655
Summary:
This fixes a two problems:
* A mistake in `KeyedMutex.runExclusive`.
* Logic about saving a document to s3 when the document is found to match what is already there.
`HostedStorageManager.flushDoc` could get caught in a loop if a document was uploaded to s3 and then, without any change to it, marked as dirty. Low level code would detect there was no change and skip the upload; but then the snapshotId could be unknown, causing an error and retries. This diff fixes that problem by discovering the snapshotId on downloads and tracking it. It also corrects a mutex problem that may have been creating the scenario. A small delay is added to `flushDoc` to mitigate the effect of similar problems in future. Exponential backoff would be good, but `flushDoc` is called in some situations where long delays would negatively impact worker shutdown or user work.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2654
Summary:
After an import from inside a document, one minute later, an important
QuerySet would get disposed, leaving the view section in a bad state,
and manifesting as JS errors on subsequent operations. (Might not
*always* happen because switching pages would prevent it from
manifesting, I think.)
Bad state that I've seen after transforms is probably explainable as
this bug, which is unrelated. Reproduction was hard because who knew one
had to wait a minute?!
Test Plan:
Added a unittest for the fix in QuerySet, and a browser test that
fails without the fix (JS errors, bad state), and passes with.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2653
Summary:
- Introduce a new SQLiteDB migration, which adds DB columns for formula columns
- Newly added columns have the special ['P'] (pending) value in them
(in order to show the usual "Loading..." on the first load that triggers the migration)
- Calculated values are added to .stored/.undo fields of user actions.
- Various changes made in the sandbox to include .stored/.undo in the right order.
- OnDemand tables ignore stored formula columns, replacing them with special SQL as before
- In particular, converting to OnDemand table leaves stale values in those
columns, we should maybe clean those out.
Some tweaks on the side:
- Allow overriding chai assertion truncateThreshold with CHAI_TRUNCATE_THRESHOLD
- Rebuild python automatically in watch mode
Test Plan: Fixed various tests, updated some fixtures. Many python tests that check actions needed adjustments because actions moved from .stored to .undo. Some checks added to catch situations previously only caught in browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2645
Summary: Adds a granular access clause for columns. Permissions can be specified for a set of columns within a table. Permissions accumulate over clauses, in a way that is intended as a placeholder pending final design.
Test Plan: Added tests. Tested manually that updates to private columns are not sent to people who don't have access to them. There are a lot of extra tests needed and TODOs to be paid down after this experimental phase.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2651
Summary:
In an emergency, we may want to serve certain documents with "old" workers as we fix problems. This diff adds some support for that.
* Creates duplicate task definitions and services for staging and production doc workers (called grist-docs-staging2 and grist-docs-prod2), pulling from distinct docker tags (staging2 and prod2). The services are set to have zero workers until we need them.
* These new workers are started with a new env variable `GRIST_WORKER_GROUP` set to `secondary`.
* The `GRIST_WORKER_GROUP` variable, if set, makes the worker available to documents in the named group, and only that group.
* An unauthenticated `/assign` endpoint is added to documents which, when POSTed to, checks that the doc is served by a worker in the desired group for that doc (as set manually in redis), and if not frees the doc up for reassignment. This makes it possible to move individual docs between workers without redeployments.
The bash scripts added are a record of how the task definitions + services were created. The services could just have been copied manually, but the task definitions will need to be updated whenever the definitions for the main doc workers are updated, so it is worth scripting that.
For example, if a certain document were to fail on a new deployment of Grist, but rolling back the full deployment wasn't practical:
* Set prod2 tag in docker to desired codebase for that document
* Set desired_count for grist-docs-prod2 service to non-zero
* Set doc-<docid>-group for that doc in redis to secondary
* Hit /api/docs/<docid>/assign to move the doc to grist-docs-prod2
(If the document needs to be reverted to a previous snapshot, that currently would need doing manually - could be made simpler, but not in scope of this diff).
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2649
Test Plan: Added some checks that company and use_school get recorded. (Live doc will need to be updated before release.)
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2650
Summary:
Deliberate changes:
* save snapshots to s3 prior to migrations.
* label migration snapshots in s3 metadata.
* avoid pruning migration snapshots for a month.
Opportunistic changes:
* Associate document timezone with snapshots, so pruning can respect timezones.
* Associate actionHash/Num with snapshots.
* Record time of last change in snapshots (rather than just s3 upload time, which could be a while later).
This ended up being a biggish change, because there was nowhere ideal to put tags (list of possibilities in diff).
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2646
Summary:
- When displaying, include the country code, and don't assume state is always present.
- When entering, include a country selector (defaulting to US), and
make state/zip optional when non-US.
- Bring in an npm module with country codes.
Test Plan: Added a browser test case.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2647
Summary:
This is a prototype for expanding the conditions that can be used in granular ACLs.
When processing ACLs, the following variables (called "characteristics") are now available in conditions:
* UserID
* Email
* Name
* Access (owners, editors, viewers)
The set of variables can be expanded by adding a "characteristic" clause. This is a clause which specifies:
* A tableId
* The name of an existing characteristic
* A colId
The effect of the clause is to expand the available characteristics with all the columns in the table, with values taken from the record where there is a match between the specified characteristic and the specified column.
Existing clauses are generalized somewhat to demonstrate and test the use these variables. That isn't the main point of this diff though, and I propose to leave generalizing+systematizing those clauses for a future diff.
Issues I'm not dealing with here:
* How clauses combine. (The scope on GranularAccessRowClause is a hack to save me worrying about that yet).
* The full set of matching methods we'll allow.
* Refreshing row access in clients when the tables mentioned in characteristic tables change.
* Full CRUD permission control.
* Default rules (part of combination).
* Reporting errors in access rules.
That said, with this diff it is possible to e.g. assign a City to editors by their email address or name, and have only rows for those Cities be visible in their client. Ability to modify those rows, and remain updates about them, remains under incomplete control.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2642
Summary:
* Fix old download endpoint to correctly pass org info in redirect.
* Switch to use newer download endpoint in client.
Old endpoint not removed. I started doing that, but it is used in copying, and it struck me that I'm not sure what should happen when copying from a site document to "Personal" - should it be the Personal that is associated with docs.getgrist.com currently, of should it be the Personal that is associated with the email of the user on whatever-site-we-are-on.getgrist.com. So leaving that as separate work.
Test Plan: updated tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2639
Summary:
- Add a /welcome/info endpoint, to serve a page after /welcome/user
- Add a new forms module to factor out the styles that feel more natural for a web form.
- Simplify form submission using JSON with a BaseAPI helper.
- The POST submission to /welcome/info gets added to a Grist doc, using a
specialPermit grant to gain access. A failure (e.g. missing doc) is logged
but does not affect the user.
Test Plan: Added a test case.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2640
Summary:
- Include column headers on each page for printing tables.
- Avoid page-breaks inside rows or cards of a card-list.
- Fix printing of charts that did not show up at all before.
- Add a browser test, not great, but somewhat functional.
Test Plan: New test, plus tested manually. Column headers work on Chrome and Firefox (not Safari).
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2636
Summary:
This implements a form of row-level access control where for a
given table, you may specify that only owners have access to
rows for which a given column has falsy values.
For simplicity:
* Only owners may edit that table.
* Non-owners with the document open will have forced
reloads whenever the table is modified.
Baby steps...
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2633
Summary:
- Supports multi-page printing with some aggressive css overrides.
- Relies on a new function implemented by grist-plugin-api to print a
multi-page CustomView.
- Renders all rows for printing for scrolly-based views.
Test Plan:
Doesn't seem possible to do a selenium test for printing. Tested
manually on Chrome, Firefox, and Safari.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2635
Summary:
- Move logo.css to core, since it's not included otherwise
- Remove unused old DocList and ViewLinker files.
- Remove #grist-app div that was only serving to supply a background
Test Plan: No changes of behavior, existing tests should pass.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2634
Summary:
For methods other than `GET`, `HEAD`, and `OPTIONS`, allow cookie-based authentication only if a certain custom header is present.
Specifically, we check that `X-Requested-With` is set to `XMLHttpRequest`. This is somewhat arbitrary, but allows us to use https://expressjs.com/en/api.html#req.xhr.
A request send from a browser that sets a custom header will prompt a preflight check, giving us a chance to check if the origin is trusted.
This diff deals with getting the header in place. There will be more work to do after this:
* Make sure that all important endpoints are checking origin. Skimming code, /api endpoint check origin, and some but not all others.
* Add tests spot-testing origin checks.
* Check on cases that authenticate differently.
- Check the websocket endpoint - it can be connected to from an arbitrary site; there is per-doc access control but probably better to lock it down more.
- There may be old endpoints that authenticate based on knowledge of a client id rather than cookies.
Test Plan: added a test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2631
Summary:
Invite links broke when some base domain plumbing changed.
This fix updates them to be aware of the base domain,
and tests the Notifier class with APP_HOME_URL set to
make sure the environment variable has the expected effect.
Test Plan: added test, updated tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2630
Summary: On Safari, in particular on mobile, the custom widget was truncated in height.
Test Plan: Tested manually on FF, Chrome, Safari (desktop) and Safari, Firefox (mobile)
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2629
Summary:
- Factor out focusing logic from Clipboard to FocusLayer.
- Generalize FocusLayer to support adding a temporary layer while a modal is open.
- Stop Mousetrap shortcuts while a modal is open.
- Refactor how Mousetrap's custom stopCallback is implemented to avoid
needing to bundle knockout for mousetrap.
Test Plan: Added a test that Enter in a UserManager doesn't open a cell editor from underneath the modal.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2626
Summary:
This moves all client code to core, and makes minimal fix-ups to
get grist and grist-core to compile correctly. The client works
in core, but I'm leaving clean-up around the build and bundles to
follow-up.
Test Plan: existing tests pass; server-dev bundle looks sane
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2627
Summary:
AccessRules class that implements that UI is intended to look vaguely like
detailed rules might look in the future, but only supports the very limited set
we have now.
In addition, UserManager and BillingPage code is separated into their own webpack bundles, to reduce the sizes of primary bundles, and relevant code from them is loaded asynchronously.
Also add two TableData methods: filterRowIds() and findMatchingRowId().
Test Plan: Only tested manually, proper automated tests don't seem warranted for this temporary UI.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2620
Summary: This removes some old metric code. There's also a user preference dialog that has a single option (whether to allow metrics) this is left in place with a dummy option. It could be ripped out as well, probably.
Test Plan: existing tests pass
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2622
Summary:
Render simple differences between documents.
* Show cell changes.
* Show cell conflicts.
* Show row additions/deletions.
Doesn't support any schema changes, and is untested in the presence of schema changes. Any widgets that access row data without using `cells` fields won't receive correct data.
Not addressed:
* Rendering conflicts in mixed row addition/updating/deleting.
* Column additions/deletions, option changes, etc.
* Document level changes.
* Table and column renames (though anticipated in ActionSummary structure).
* Page-level changes.
* Drawing attention to changes (marking changed pages+views, suppressing
unchanged rows, etc).
* Rendering differences in views other than GridView.
* Adding UI for initiating a comparison.
* Editing while comparing.
Replaces {D2600}
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2618
Summary:
- Show a 'Copy Link' button in UserManager.
- Add icons for Copy (to copy link), and also for Video (to open video
tutorials in later onboarding changes)
- Add to UserManager a 'Public Access' member-like line for greater visibility
and to allow changing role.
- In main document page, add a "public access" icon.
- On saving UserManager, re-fetch DocInfo to update "public access" icon.
Test Plan: TBD
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2617
Summary:
If a `compare` query parameter is supplied, this diff will load
the difference between the referenced document and the current
document into an observable in the GristDoc.
Nothing is done with the comparison yet. Comparisons are not
yet live - they don't get updated if either document changes.
For convenience, `window.gristDocPageModel` is set as an easy
way to access the DocPageModel from the browser console.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2615
Summary:
Add a unittest that start the server with limited memory, and build just enough
of ActionHistory to crash the server before this fix, and not after.
Test Plan: Tested manually with various memory prints, and added a test.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2616
Summary:
- The older "Anonymous access" option is renamed to "Public access" in the UI.
- This option was only present with GRIST_SUPPORT_ANON set.
- With GRIST_SUPPORT_ANON, the old behavior is kept: "Public access"
option adds/removes anon@ user
- Without GRIST_SUPPORT_ANON (normal case), orgs/workspaces don't support
"Public access" option. For documents, it adds/removes everyone@ user.
The latter is the main feature of interest. The GRIST_SUPPORT_ANON flag
is set for on-premise installs, and adds discoverability by anon users.
Test Plan: Added a test cases and checks in other tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2612
Summary:
* Extends `/api/docs/docId1/compare/docId2` endpoint with a `detail=1` option to include details of what changed in the document content.
* Adds an `/api/docs/docId/compare?left=HASH&right=HASH` endpoint for comparing two versions of a single document. This is needed to implement the extension to `/api/docs/docId1/compare/docId2`.
* Adds a `HashUtil` class to allow hash aliases like `HEAD` and `HEAD~`.
Everything is a bit crude:
* Changes are expressed as ActionSummary objects, which aren't fully fleshed out.
* Extra data about formula columns is inserted in an inflexible way.
This is extracted and cleaned up from https://phab.getgrist.com/D2600.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2614
Summary:
This is an incremental step in granular access control. Using
a temporary `{colIds: '~o structure'}` representation in the
`_grist_ACLResources` table, the document structure can be set
to be controlled by owners only.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2613
Summary:
This ports two classes touched by data-diffing branch to typescript, so that the code is easier to understand and modify.
DataRowModel is quite entangled with its base class, but porting it too got a little complicated.
Test Plan: existing tests pass
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2611
Summary:
This makes it possible to serve a table or tables only to owners.
* The _grist_ACLResources table is abused (temporarily) such that rows of the form `{colId: '~o', tableId}` are interpreted as meaning that `tableId` is private to owners.
* Many websocket and api endpoints are updated to preserve the privacy of these tables.
* In a document where some tables are private, a lot of capabilities are turned off for non-owners to avoid leaking info indirectly.
* The client is tweaked minimally, to show '-' where a page with some private material would otherwise go.
No attempt is made to protect data from private tables pulled into non-private tables via formulas.
There are some known leaks remaining:
* Changes to the schema of private tables are still broadcast to all clients (fixable).
* Non-owner may be able to access snapshots or make forks or use other corners of API (fixable).
* Changing name of table makes it public, since tableId in ACLResource is not updated (fixable).
Security will require some work, the attack surface is large.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2604
Summary:
- The card includes an image, a brief description, and a link to the tutorial.
- The left panel includes a link to the tutorial, and a button to reopen card.
- Card is collapsed and expanded with a little animation.
- Add a seenExamples pref for whether an example has been seen.
- Store the pref in localStorage for anon user.
Separately, added clearing of prefs of test users between tests, to avoid tests
affecting unrelated tests.
Test Plan: Added a browser test.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2602
Summary: This makes the user's role (owner/editor/viewer) available in ActiveDoc methods. No use of that information is made yet, other than to log it. The bulk of the diff is getting a handle on the various ways the methods can be called, and systematizing it a bit more. In passing, access control is added to broadcasts of document changes, so users who no longer have access to a document do not receive changes if they still have the document open.
Test Plan: existing tests pass; test for broadcast access control added
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2599
Summary:
- Change RECORD's dates_as_str default to False.
- Reimplement objtype encode_object/decode_object with less machinery.
- Implement encoding of dicts (with string keys).
- Make lists and dicts encode values recursively.
- Implement encoding/decoding in the client
- Decode automatically in plugins' fetchSelectedTable/Record, with an option to skip.
Test Plan: Tested manually, not sure what tests may be affected yet.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2593
Summary:
- Add icons to indicate a publicly-accessible document
- Dim inaccessible DocMenu items
Test Plan: Added a browser test for rendering inaccessible and public docs
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2594
Summary:
Sharing a document with everyone@ was effective at the api level,
but had two flaws in the web client:
* A logged in user with no access at the org level could not access
a publically shared doc within that org.
* Likewise, for the anonymous user (but for a different reason).
This diff tweaks the web client to permit accessing a doc when
org information is unavailable.
It also changes how redirects happen for the anonymous user when
accessing a doc. They now only happen once it has been confirmed
that the user does not have access to the doc.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2591
Summary:
- Add org-wide currentSort and currentView, saved as user preferences.
- Add per-workspace currentSort and currentView, backed by localStorage.
- Move localStorage-based observables to a separate file.
- Move hard-coded data about example docs to a separate file.
- Add UI for toggling sort and view mode.
- Removed unused features of buttonSelect to simplify it,
and added support for light style of buttons.
- Added `parse` helper method to StringUnion, and use it in a few places where
it simplifies code.
- Set `needRealOrg: true` in HomeDBManager.updateOrg() to fix saving prefs for
mergedOrg.
Test Plan: WIP: Fixed some affected tests. New tests not yet written.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2587
Summary:
This adds query parameters useful for tailoring the Grist experience, with an eye to embedding.
Setting `style=light` removes side and top bars, as a first pass at a focused view of a single document page (this would benefit from refining).
Setting `embed=true` has no significant effect just yet other than it restricts document access to viewer at most (this can be overridden by specifying `/m/default`).
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2585
Summary:
- Instead of sending an "UnmarshallableError" as an exception, introduce an
"Unmarshallable" type of value, represented as ['U', repr(value)]
- Unmarshallable values are rendered using a bluish text color, no longer a
pink background.
- Factor out ErrorDom to be simpler and cleaner.
- Add GristObjCode enum, and simplify related helpers.
- Use safe_repr() for when repr() itself fails
- Handle conversion errors using safe_repr() when str() fails
Test Plan: Added a test case based on a fixture covering a bunch of cases.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2584
Summary:
This simplifies writing custom widgets that access selected
data. To access the record at which the cursor is set, and
get any future changes to it as the cursor moves or data
changes, it suffices now to do:
```
grist.ready();
grist.onRecord(record => /* render */);
```
Similarly to access the set of selected records, and get any
changes, it suffices now to do:
```
grist.ready();
grist.onRecords(records => /* render */);
```
The `records` argument will be a list of objects, each of which
is a single record. This is distinct from the column-based
representation favored in Grist up ontil now. That remains
how methods like `fetchTable` or `fetchSelectedTable` represent
their results. In the future, methods named like `fetchRecords`
or `fetchSelectedRecords` could be added that return lists.
Test Plan: extended tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2583
Summary:
- Hides left and right panels and the top bar
- Hides cursor and active-section highlight
- Hides "=" icon on formulas
- Nudges browser to include background for row/column headers, which is not
otherwise included.
Still only what's visible is printed (e.g. large tables not paginated).
Test Plan: Tried a few pages manually on Firefox and Chrome.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2579
Summary:
When listing access on a doc or workspaces, include all users associated with
the resource or its parents.
Previously we only considered org-level users. This is normally sufficient
since doc and workspace users are automatically added as guests of the org. But
there are exceptions for special users (like everyone@), and generally, in case
of any divergence, it's important to list everyone who affects access
decisions.
Test Plan: Added a test that everyone@ user gets included in listings
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2533
Summary:
Only allow authorized users to set names. This excludes the anonymous user.
Seems to be a narrow issue isolated to the `POST /api/profile/user/name` endpoint, other `profile` posts/deletes are already restricted to authorized users.
Test Plan: added a test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2574
Summary:
Adds preferences to orgs. There are a few flavors:
* `userOrgPrefs`: these are specific to a certain user and a certain org.
* `orgPrefs`: these are specific to a certain org, and apply to all users.
* `userPrefs`: these are specific to a certain user, and apply to all orgs.
The three flavors of prefs are reported by `GET` for an org, and can be modified by `PATCH` for an org. The user needs to have UPDATE rights to change `orgPrefs`, but can change `userOrgPrefs` and `userPrefs` without that right since the settings only affect themselves.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2572
Summary:
1. The /import endpoint wasn't handling poor names like ".grist" as
intended, instead trying to import them using the plugin-based imports.
2. The SaveCopy dialog was allowing users to save to an empty name,
particularly bad because new docs now default to an empty name.
Error manifested as "Cannot parse data" to the user.
Reported in https://secure.helpscout.net/conversation/1242629116/292
Test Plan: Added tests for both parts of the fix.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2573
Summary:
* Checks that empty workspaces are listed correctly, including in
cases where docs or workspaces have been made inaccessible to
the user doing the listing.
* Checks that when a document quota is in force, the count is
correct, and not dependent on ACLs.
* Fixes the document count used for document quotas, which in
fact was not counting docs the current user did not have access
to.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2570
Summary:
This modifies the material listed in workspaces. Previously,
material the user did not have access to was omitted. Now, it
is included if the user has the right to delete the workspace.
This is to avoid scenarios where a user might try to delete a
workspace without being aware of the full consequences.
Test Plan: added tests; existing tests should pass
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2568
Summary:
this moves sandbox/grist to core, and adds a requirements.txt
file for reconstructing the content of sandbox/thirdparty.
Test Plan:
existing tests pass.
Tested core functionality manually. Tested docker build manually.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2563
Summary:
- Implemented selecting an org in some cases when using Save-Copy dialog.
- Unified previous 'Save Copy' menu into an enhanced "Share" menu.
- Renamed ExportMenu to ShareMenu, collect related code into it, and design the share button.
- Introduced trunkAccess property for forks, to know whether "Replace Original" is available.
- Simplified handling of fork() result, now that all code has been upgraded.
- Replaced 'Copy as Template' menu items with a checkbox in the Save-Copy dialog
- Removed copy links for examples in the DocMenu (to simplify, since not part of updated design)
- Updated the UI of the copying dialog.
Test Plan: Updated affected tests, added new test cases for copying when other orgs are a choice or not.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2561
Summary:
* Remove duplicate schema file
* Move version file to a stub in grist-core
* Simplify sandbox creation in grist-core (although not functional until sandbox code moved)
* Add a minimal test for buildability
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2560
Summary:
This makes core independently buildable again, and adds a small
script to run as a sanity check.
Test Plan: checked that build_core.sh succeeds
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2558
Summary:
* Adds a simple deployment test for the "Import from URL" button.
* Makes server aware of plugin hostnames in the appropriate places.
* Unrelated but convenient: allows following redirection when importing.
Test Plan:
Added tests. The `local_deployment` test works. A modified
version of this works against `staging_deployment` (using a test url that
doesn't require redirection; also staging currently has a hot fix that can
hopefully be removed once the code fix included here is in).
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2556
Summary:
Currently if I share a doc with a friend, and then soft-delete a doc
in the same workspace, that friend will see the workspace in their
trash (empty, but there).
This adds a test for the issue and resolves it by filtering out
docs at the sql level that used to be filtered out by javascript.
Test Plan: added test; existing tests pass
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2557
Summary:
Give specialPermit to the support user for page loads and API requests needed
to serve billing pages.
Test Plan: Added new test cases
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2554
Summary:
The docker image was not building or running correctly
after breaking out more material into core. This corrects
the necessary paths.
Test Plan:
tested by building locally with `./contain.sh ./build docker`
and running containers in development and production mode.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2555
Summary:
- Bad status of a subscription is now reported, along with the last payment error, if any.
- Error caused when getting valueRemaining of a subscription in a bad state is now ignored.
- Certain kinds of errors from Stripe are now reported to the user in a
friendlier way (avoiding statusCode of 500 when there is a better one)
- A wide range of Stripe errors are logged with metadata.
- Show a link to the Stripe-hosted last invoice, which seems useful generally,
and also gives the user more options to pay after a payment failure.
- Get default_source along with customer to save a roundtrip to Stripe.
- Use a simpler (single) Stripe call for updating a customer’s card.
- Retry paying an invoice when updating a card when there is an unpaid invoice
with a payment error.
Some refactoring included:
- Simplified ISubscriptionModel by extending IBillingSubscription.
- Factor out common portions from several Billing tests.
Add a test case for how some card errors are reported
Add bits to the new billing test, still WIP.
Test Plan:
Added a test suite for incomplete and past_due subscriptions, and a
test case one for better error reporting.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2553
Summary: This moves enough server material into core to run a home server. The data engine is not yet incorporated (though in manual testing it works when ported).
Test Plan: existing tests pass
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2552
The cssVars.ts file has changed to include some more knobs
for custom theming. This commit updates the file, and
introduces a `stubs` directory for stubbing code that is
specific to our deployments of Grist and not of general interest.
