Summary:
The old Examples and Templates workspace is now
a page that pulls templates from a new public Grist Templates org.
The All Documents view will pull featured templates from that org, where
featured templates are simply pinned documents in Grist Templates. The
Examples and Templates page will also show the featured templates, as
well as the rest of the available templates organized by category. The
categories are equivalent to workspaces in Grist Templates, and are
generated dynamically.
Test Plan: Browser tests.
Reviewers: paulfitz, dsagal
Reviewed By: paulfitz, dsagal
Subscribers: dsagal, paulfitz, jarek
Differential Revision: https://phab.getgrist.com/D2930
Summary:
* Moves essential plugins to grist-core, so that basic imports (e.g. csv) work.
* Adds support for a `GRIST_SANDBOX_FLAVOR` flag that can systematically override how the data engine is run.
- `GRIST_SANDBOX_FLAVOR=pynbox` is "classic" nacl-based sandbox.
- `GRIST_SANDBOX_FLAVOR=docker` runs engines in individual docker containers. It requires an image specified in `sandbox/docker` (alternative images can be named with `GRIST_SANDBOX` flag - need to contain python and engine requirements). It is a simple reference implementation for sandboxing.
- `GRIST_SANDBOX_FLAVOR=unsandboxed` runs whatever local version of python is specified by a `GRIST_SANDBOX` flag directly, with no sandboxing. Engine requirements must be installed, so an absolute path to a python executable in a virtualenv is easiest to manage.
- `GRIST_SANDBOX_FLAVOR=gvisor` runs the data engine via gvisor's runsc. Experimental, with implementation not included in grist-core. Since gvisor runs on Linux only, this flavor supports wrapping the sandboxes in a single shared docker container.
* Tweaks some recent express query parameter code to work in grist-core, which has a slightly different version of express (smoke test doesn't catch this since in Jenkins core is built within a workspace that has node_modules, and wires get crossed - in a dev environment the problem on master can be seen by doing `buildtools/build_core.sh /tmp/any_path_outside_grist`).
The new sandbox options do not have tests yet, nor does this they change the behavior of grist servers today. They are there to clean up and consolidate a collection of patches I've been using that were getting cumbersome, and make it easier to run experiments.
I haven't looked closely at imports beyond core.
Test Plan: tested manually against regular grist and grist-core, including imports
Reviewers: alexmojaki, dsagal
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2942
Summary:
Extracts code from showExampleCard into a generic function which is reused for document tours.
It handles reading and writing to user preferences for automatic showing and explicitly reopening.
Test Plan:
Manually tested that it automatically shows a tour just once and clicking to reopen works.
There's not much new functionality so there's little that needs testing. This is an initial version that's mostly internal and is likely to be polished for users in the future.
If I should still add tests, I'd like confirmation that the current behaviour is as desired.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2944
Summary:
Like the welcome tour, a special URL hash triggers startDocTour which uses data from a table GristDocTour to construct the appropriate popups.
This is the basic version described in https://grist.quip.com/sN2RAHI2dchm/Document-tours
Test Plan:
Added a new nbrowser test which tests the data produced by makeDocTour. The general behaviour of the UI and popups has hardly changed so existing tests cover that well enough.
The new test uses a new fixture document which you can open to easily experience the tour.
Error cases where there's no valid document tour are not tested because that behaviour is likely to change significantly and this feature is still quite 'private'.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: jarek, dsagal
Differential Revision: https://phab.getgrist.com/D2938
Summary:
This makes it possible to set the type of a column to ReferenceList, but the UI is terrible
ReferenceList.ts is a mishmash of ChoiceList and Reference that sort of works but something about the CSS is clearly broken
ReferenceListEditor is just a text editor, you have to type in a JSON array of row IDs. Ignore the value that's present when you start editing. I can maybe try mashing together ReferenceEditor and ChoiceListEditor but it doesn't seem wise.
I think @georgegevoian should take over here. Reviewing the diff as it is to check for obvious issues is probably good but I don't think it's worth trying to land/merge anything.
Test Plan: none
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D2914
Summary:
Flaky Dates test failures related to the use of JQuery autocomplete for time
zones, which wasn't working well.
This diff replaces that autocomplete (as well as a similar select box in
DocumentSettings) with our newer autocomplete, adding some select-box like
behavior.
Most of the behavior is factored out into ACSelect, which could be more
generally useful.
Adds an option to autocomplete to keep options ordered according to their
initial order.
Unrelated: fix up usage of MultiHolder in Drafts to avoid 'already disposed'
warnings.
Test Plan: Fixed several affected tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D2919
Summary: Last document position was overwritting anchor link navigation.
Test Plan: Browser tests
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D2934
Summary:
Fixing two bugs
- Google Auth Endpoint wasn't resolving protocol in a correct way
- Google Auth Popup was navigationg to endpoint url based on home url, which
was diffent from current page origin
Test Plan: n/a
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2937
Summary:
This tweaks pre-fork mode to make the user's experience a bit more seamless.
Pre-fork mode is where the user has opened a document with intent to
fork it, but actual forking (with allocation of a new document id)
is postponed until they make their first change.
The tweak makes the user an owner for granular access purposes, if
forking is permitted. So data visible only to owners because of
access rules will be visible to them. As always, any edits would
go to a separate new copy.
A remaining tricky corner is what to do about "View As" functionality
on forks. Fork sharing cannot be controlled, so the "Users -> View As"
functionality isn't available. Perhaps the "Users" button on a fork
could encourage doing a save-copy and inviting users, or offer some
dummy users? In any case, this diff doesn't change anything with
that corner.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2931
Summary:
Google Auth popup wasn't able to resolve origin from gristConfig.
Moving this reponsability to server side, where it gets calculated from initial request.
Test Plan: n/a
Reviewers: dsagal, paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2935
Summary:
Implementing export to excel and send to Google Drive feature.
As part of this feature few things were implemented:
- Server side google authentication exposed on url: (docs, docs-s, or localhost:8080)/auth/google
- Exporting grist documents as an excel file (xlsx)
- Storing exported grist document (in excel format) in Google Drive as a spreadsheet document.
Server side google authentication requires one new environmental variables
- GOOGLE_CLIENT_SECRET (required) used by authentication handler
Test Plan: Browser tests for exporting to excel.
Reviewers: paulfitz, dsagal
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2924
Summary: Adding a new element to a new Choice or ChoiceList column resulted in javascript error
Test Plan: Browser tests
Reviewers: georgegevoian, dsagal
Reviewed By: georgegevoian, dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2932
Summary: Remove repl.py, REPLTab.js, some wiring code, CSS, and a test in testscript.json.
Test Plan: NA
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2923
Summary:
- Fix flaky SamplesWS test, which was occasionally clicking a disabled button
- Increase timeout on backupSqliteDatabase test, which sometimes times out
- Fix a little flakiness in ExportSection test.
- Fix flaky Billing test by adjusting behavior slightly.
In Billing, when re-fetching subscription (e.g. on navigating back), we now
unset it while the fetch is pending, so that billing pages show a spinner or
"Fetching..." messages. This also gives tests something to wait for.
Also adjusts Billing styles on the "Fetching..." messages to make them look
different from regular text.
Test Plan: Mainly making exising tests more robust. Billing changes exercised by existing tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2920
Summary:
This diff brings in the new welcome tour. It builds upon `client/ui/OnBoardingPopup` that was committed to that purposes. Per this diff, the tour is accessible behind a flag and won't be visible to user: few caveats listed below needs to be adressed first.
This diff also brings few changes to onboarding module.
- allow to refer to element with selector
- usually dynamic selection of element sounds useful for when the
element does not exist yet when the tour starts. But the actual
reason when add it here, is to allow selecting the first cell.
- if the selector yields undefined (missing element), the popup
is simply skipped
- got rid of the internal registry to link between popup contents
and popup options. All is now define in the same interface. Registry
overall felt overkill and not needed.
- adds an option to show message as a simple modal that is centered
on the screen
This diff also brings the new welcome tour and hide it behind a flag
CAVEATS that need to be addressed in follow up commit:
- The url needs cleanup, #repeat-welcome-tour sticks to it and so even when navigating to home page. This could eventually become an issue: if user opens another document it would starts the onboarding tour again.
- For now you have to manually make sure the right panel is opened with the Column tab selected before starting the tour.
- On boarding tours were not designed with mobile support in mind. So probably a good idea to disable.
- Backend support needs to be done (persistence of first time user).
Test Plan:
Updated `projects/OnBoardingPopup` and adds new `nbrowser/welcomeTour`
To launch the tour:
- open any document
- open manually the right panel and the field tab
- append the flag `#repeat-welcome-tour` at the end of the url in the url bar and reload the page
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2917
Summary:
Choice columns can now add new choices directly
from the autocomplete menu. The autocomplete will
now highlight the first matching item, even if there are equally
ranked alternatives. No changes have been made to how the
autocomplete index is created, or how it scores items.
For choice and choice list columns, the filter menu will
now display values using their configured colors, similar to the
rest of the UI. Choice tokens throughout the UI now do a better
job of handling text overflow by showing an ellipsis whenever
there isn't enough space to show the full text of a choice.
Test Plan: Browser tests.
Reviewers: cyprien
Reviewed By: cyprien
Differential Revision: https://phab.getgrist.com/D2904
Summary:
Bundles some new document options into a JSON column.
The icon option is treated somewhat gingerly. It is intended, at
least initially, to store an image thumbnail for a document as a
url to hand-prepared assets (for examples and templates), so it is
locked down to a particular url prefix to avoid opening the door to
mischief.
Test Plan: added test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D2916
Summary:
The 'user' variable has a similar API to the one from access rules: it
contains properties about a user, such as their full name and email
address, as well as optional, user-defined attributes that are populated
via user attribute tables.
Test Plan: Python unit tests.
Reviewers: alexmojaki, paulfitz, dsagal
Reviewed By: alexmojaki, dsagal
Subscribers: paulfitz, dsagal, alexmojaki
Differential Revision: https://phab.getgrist.com/D2898
Summary:
When redirecting to login, it's important to have a valid session set. This was
done by middleware that only applies to home pages. We need to set session to
live when redirecting in case of doc pages too.
Test Plan: Added a test case for fixed behavior by applying an existing case to doc pages too
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2915
Summary:
This suspends service to a team site for which an AppSumo refund has been made, and nudges users to their free personal account.
I expect that a refund request would fail for a site where user is also paying us for extra seats.
Test Plan: tested manually
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2912
Summary:
This applies some mitigations suggested by SQLite authors when
opening untrusted SQLite databases, as we do when Grist docs
are uploaded by the user. See:
https://www.sqlite.org/security.html#untrusted_sqlite_database_files
Steps implemented in this diff are:
* Setting `trusted_schema` to off
* Running a SQLite-level integrity check on uploads
Other steps will require updates to our node-sqlite3 fork, since they
are not available via the node-sqlite3 api (one more reason to migrate
to better-sqlite3).
I haven't yet managed to create a file that triggers an integrity
check failure without also being detected as corruption by sqlite
at a more basic level, so that is a TODO for testing.
Test Plan:
existing tests pass; need to come up with exploits to
actually test the defences and have not yet
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2909
Summary:
Finishing implementation for google drive plugin.
- Refactoring plugin code to make it more robust and to follow grist ux
- Changing the way server hosts untrusted user content, from different domain to different port
Test Plan: Browser tests
Reviewers: dsagal, paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2881
Summary:
Error notifications include a "Report a problem" link, but the inclusion of
error message and stack trace was broken.
Test Plan: Tested manually and added a test case.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2911
Summary:
The sigma icon appears to the left of the table title
if the table is a summary table.
Test Plan: Updated browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D2895
Summary:
API signature for autocomplete updated to add column ID, which is
necessary for exposing correct types for 'value'.
Test Plan: Unit tests.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: jarek, alexmojaki
Differential Revision: https://phab.getgrist.com/D2896
Summary:
In the past, Cognito sign-ins were intended to give authorization to some AWS
services (like SQS); various tokens were stored in the session for this
purpose. This is no longer used. Profiles from Cognito now serve a limited
purpose: first-time initialization of name and picture, and keeping track of
which login method was used. For these remaining needs, ScopedSession is
sufficient.
Test Plan:
Existing test pass. Tested manually that logins work with Google and
Email + Password. Tested manually that on a clean database, name and picture
are picked up from a Google Login.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2907
Test Plan: Checked manually for a long-opening document that the time reported is correct.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2906
Test Plan: Existing tests should pass, no new ones needed
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2905
Summary:
This switches to using stdin/stdout for RPC calls to the sandbox, rather than specially allocated side channels. Plain text error information remains on stderr.
The motivation for the change is to simplify use of sandboxes, some of which support extra file descriptors and some of which don't.
The new style of communication is made the default, but I'm not committed to this, just that it be easy to switch to if needed. It is possible I'll need to switch the communication method again in the near future.
One reason not to make this default would be windows support, which is likely broken since stdin/stdout are by default in text mode.
Test Plan: existing tests pass
Reviewers: dsagal, alexmojaki
Reviewed By: dsagal, alexmojaki
Differential Revision: https://phab.getgrist.com/D2897
Summary:
Includes overhauled choice configuration UI for choice and choice list
columns based on the TokenField library. Features include rich copy
and paste support, keyboard shortcuts for token manipulation, and
drag-and-drop support for arrangement.
Configured choice colors are visible throughout the application, such
as in the autocomplete window for both choice and choice list cells, and
in table cells directly.
Choice cells in particular are now styled closer to choice list cells,
and render their contents as colored tokens. Choice cells now also
use the improved autocomplete component that choice lists use, with
some room for future improvement (e.g. allowing new choice items to be
added inline like in choice list's autocomplete).
Also includes a minor fix for choice list cells where right align
was not working.
Test Plan: Browser tests updated.
Reviewers: jarek, dsagal
Reviewed By: jarek, dsagal
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D2890
Summary:
- Normally Reference columns can only be used for data entry once the target
table has loaded. When it shows RowID, we shouldn't need to wait.
- Also, fix pasting values between cells of a RowID-showing column.
Test Plan: Added a test for entering data before data has loaded.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2902
Summary:
Branding feedback from AppSumo found a capitalization problem. They also nudged us again to include a link back for the user to manage
their AppSumo account.
Test Plan: manual
Reviewers: dsagal, anaisconce
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2901
Summary:
Does the UI only no backend.
Follow up work:
- Implement a way to remember when a user dimsmis the popups, so
that we don't show her again.
- After users clicks Finish adds a final popup saying "You can repeat this tour from the Help Center" , and in help center home page, have a link "Repeat Grist welcome tour", which opens, say, https://docs.getgrist.com/doc/lightweight-crm#repeat-welcome-tour, where the hash part tells us to repeat the tour.
Test Plan: Tested in project/OnBoardingPopups
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2892
Summary: Removed test/aws/, most of app/server/lib/, 3 dirs in app/lambda/, corresponding tests, and more!
Test Plan: a lot of this is quite the opposite...
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2894
Summary:
Run JS with a value for SANDBOX_BUFFERS_DIR, then run test_replay in python with the same value to replay just the python code.
See test_replay.py for more info.
Test Plan:
Record some data, e.g. `SANDBOX_BUFFERS_DIR=manual npm start` or `SANDBOX_BUFFERS_DIR=server ./test/testrun.sh server`.
Then run `SANDBOX_BUFFERS_DIR=server python -m unittest test_replay` from within `core/sandbox/grist` to replay the input from the JS.
Sample of the output will look like this:
```
Checking /tmp/sandbox_buffers/server/2021-06-16T15:13:59.958Z
True
Checking /tmp/sandbox_buffers/server/2021-06-16T15:16:37.170Z
True
Checking /tmp/sandbox_buffers/server/2021-06-16T15:14:22.378Z
True
```
Reviewers: paulfitz, dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2866
Summary:
- Implement UI with "Apply to new records" and "Apply on record changes"
checkboxes, and options for selecting which changes to recalculate on.
- For consistency, always represent empty RefList as None
- Fix up generated SchemaTypes to remember that values are encoded.
Included test cases for the main planned use cases:
- Auto-filled UUID column
- Data cleaning
- NOW() formula for record's last-updated timestamp.
- Updates that depend on other columns.
Test Plan: Added a browser test.
Reviewers: jarek
Reviewed By: jarek
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2885
Summary:
User was not able to delete cards. This patch introduces a context menu for cards, analogous to the one available for rows on a GridView.
Changes:
- Row numbers on a GridView have the same icon as on columns to make context menu more discoverable.
- Context menu for rows and columns, when activated, didn't switch section in rare conditions (i.e. when the section had 2 or more columns selected, one of which had the same rowId as a column in the section that the user switched from).
- Card list layout and a single card layout has the same context menu as in a GridView, available by pressing the context menu button.
Test Plan: Browser tests
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2870
Summary:
Trigger formulas can be calculated for new records, or for new records and
updates to certain fields, or all fields. They do not recalculate on open,
and they MAY be set directly by the user, including for data-cleaning.
- Column metadata now includes recalcWhen and recalcDeps fields.
- Trigger formulas are NOT recalculated on open or on schema changes.
- When recalcWhen is "never", formula isn't calculated even for new records.
- When recalcWhen is "allupdates", formula is calculated for new records and
any manual (non-formula) updates to the record.
- When recalcWhen is "", formula is calculated for new records, and changes to
recalcDeps fields (which may be formula fields or column itself).
- A column whose recalcDeps includes itself is a "data-cleaning" column; a
value set by the user will still trigger the formula.
- All trigger-formulas receive a "value" argument (to support the case above).
Small changes
- Update RefLists (used for recalcDeps) when target rows are deleted.
- Add RecordList.__contains__ (for `rec in refList` or `id in refList` checks)
- Clarify that Calculate action has replaced load_done() in practice,
and use it in tests too, to better match reality.
Left for later:
- UI for setting recalcWhen / recalcDeps.
- Implementation of actions such as "Recalculate for all cells".
- Allowing trigger-formulas access to the current user's info.
Test Plan: Added a comprehensive python-side test for various trigger combinations
Reviewers: paulfitz, alexmojaki
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2872
Summary:
Current appsumo sign-up flow doesn't reach the billing pages.
This diff nudges user on through that extra step.
It also tweaks plan summaries to say what special appsumo
features are in effect (member count prepaid for).
Test Plan: manual
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2882
Summary:
This adds a new landing page for cognito sign-up, intended for
use by new appsumo users.
Their email address is pre-filled and locked down, and sign-up
is by entering a password.
The page is very crude compared to hosted cognito - especially
in error reporting! - but having the address filled in more
than makes up for that.
The flow does not quite connect with the new billing signup.
I think we can do that through the regular "welcome" process,
which will list the user's team site. When the user visits
that site, we could detect that we are on a site with no
domain set yet and for which the user is a billing manager,
and trigger a visit to the appropriate billing page.
Test Plan: manual - hard to test through cognito email step
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2880
Summary:
This links AppSumo sign-ups with Stripe subscriptions
and our billing pages. Different AppSumo tiers are supported by
different coupons on the standard plan. Configuration of this
is in stripe, and then cached in the database.
The front end is tweaked just enough to make completing a sign-up
possible. It is not yet friendly.
Not covered includes:
* Streamlining landing page.
* Making billing pages git clearer summaries of AppSumo states.
* Making flow through Cognito as graceful as possible - default
probably doesn't meet AppSumo requirements.
* Disabling site on cancellation/refund.
* Downgrades when more seats in use than lower tier allows.
Test Plan: api-level tests added. No front-end tests yet.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2878
Test Plan: Enhanced the test case for memos to check these cases too (fails without this fix).
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2876
Summary:
Makes filter counts take other column filters into account.
- Changes the summaries rows to reflect hidden rows:
- hidden rows are added to the `Other Values` summary
- show the unique number of other values as `Other Values (12)`
- Also, adds a sort button to the column filter menu
Test Plan: Adds browser test.
Reviewers: paulfitz, jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D2861
Summary:
This adds appsumo /token and /notification endpoints, with some
tests. The stub implementation is sufficient for AppSumo
activation to succeed (when exposed via port forwarding for testing).
It needs fleshing out:
* Implement upgrade/downgrade/refund and stripe subscription.
* Implement custom landing page and flow.
Test Plan: added tests
Reviewers: dsagal, georgegevoian
Reviewed By: dsagal
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D2864
Summary:
User can freeze any number of columns, which will not move when a user scrolls grid horizontally.
Main use cases:
- Frozen columns don't move when a user scrolls horizontally
- The number of frozen columns is automatically persisted
- Readonly viewers see frozen columns and can modify them - but the change is not persisted
- On a small screen - frozen columns still moves to the left when scrolled, to reveal at least one column
- There is a single menu option - Toggle freeze - which offers the best action considering selected columns
- When a user clicks a single column - action to freeze/unfreeze is always there
- When a user clicks multiple columns - action is offered only where it makes sens (columns are near the frozen border)
Test Plan: Browser tests
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2852
Summary:
Grist should not prevent read-only viewers from opening cell editors since they usually provide much more information than is visible in a cell.
Every editor was enhanced with a read-only mode that provides the same information available for an editor but doesn't allow to change the underlying data.
Test Plan: Browser tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2842
Summary:
Replaces https://phab.getgrist.com/D2854
Refactoring of NSandbox:
- Simplify arguments to NSandbox.spawn. Only half the arguments were used depending on the flavour, adding a layer of confusion.
- Ensure the same environment variables are passed to both flavours of sandbox
- Simplify passing down environment variables.
Implement deterministic mode with libfaketime and a seeded random instance.
- Include static prebuilt libfaketime.so.1, may need another solution in future for other platforms.
Recording pycalls:
- Add script recordDocumentPyCalls.js to open a single document outside of tests.
- Refactor out recordPyCalls.ts to support various uses.
- Add afterEach hook to save all pycalls from server tests under $PYCALLS_DIR
- Make docTools usable without mocha.
- Add useLocalDoc and loadLocalDoc for loading non-fixture documents
Test Plan:
Made a document with formulas NOW() and UUID()
Compare two document openings in normal mode:
diff <(test/recordDocumentPyCalls.js samples/d4W6NrzCMNVSVD6nWgNrGC.grist /dev/stdout) \
<(test/recordDocumentPyCalls.js samples/d4W6NrzCMNVSVD6nWgNrGC.grist /dev/stdout)
Output:
< 1623407499.58132,
---
> 1623407499.60376,
1195c1195
< "B": "bd2487f6-63c9-4f02-bbbc-5c0d674a2dc6"
---
> "B": "22e1a4fd-297f-4b86-91a2-bc42cc6da4b2"
`export DETERMINISTIC_MODE=1` and repeat. diff is empty!
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2857
Summary:
The loading spinner would always display
'Building Table widget' when creating or changing
a widget. This fixes the title to reflect the selected
widget type.
Test Plan:
Updated existing browser tests to verify the loading spinner
title includes the correct widget type.
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2863
Summary:
Previously, soft-deleted docs in icon view were still accessible from
the Trash and couldn't be permanently deleted.
Test Plan:
Improved the nbrowser test for deleting docs to verify that it can
be done in both view modes.
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2862
Summary: The `select-all` originally designed the `All` button but it unfortunately evolves into designating both `All` and `None`. Which is confusing. Replaces with the more general `bulk-action`.
Test Plan: Should not break anything.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2860
Test Plan: Wrote unit and browser tests that verify new behavior.
Reviewers: paulfitz, dsagal
Reviewed By: dsagal
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D2855
Summary:
- Takes advantage of native indeterminate state of html checkboxes
- When an indeterminate checkbox is clicked it turns it into being not checked.
Test Plan: - Added test to projects/UI2018
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2846
Summary:
This fixes a bug where deleting a page with the page id
missing from the URL would cause JS errors to be thrown.
Test Plan:
Verified manually in dev environment. Browser test added
that should hopefully replicate the repro steps and catch
any regressions.
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2856
Summary:
This is a somewhat experimental change, that will implement permitted parts of an undo if not all parts are permitted. This is in preparation for trigger columns, where it may become common for a change in a record resulting in a change to an automatic change to another that the user cannot edit directly. How to undo such an action is somewhat unclear. One option is to undo the permitted parts, and then the triggers can rerun.
The general case is a bit of a can of worms, and feels adjacent to merging/rebasing etc.
Oh: it would probably be important in general to communicate to the user that an undo was partial, but this diff doesn't do that. It would need some new plumbing.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2839
Summary: Cursor position observable was created using GrainJS, but the fields it was using were created using knockout observables. In a result the cursor position wasn't recomputed when a view was changed or an active section was deleted.
Test Plan: Browser tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2850
Test Plan:
Tested manually by enabling/disabling wrapping and changing alignment on reference columns. Existing test updated to check that
reference columns have cell formatting options available.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2853
Summary:
Bug summary: if in right bar user starts changing name of column, but then clicks on a different column name in table, THAT column will have its name changed.
This bug occurs because the save method is invoked by a blur event on a input field, which is triggered after all computed observables are calculated. Save method gets an observable to update, which by the time a blur event triggers, is changed to a new column.
The solution was to forcefully trigger the blur event as soon as possible - here by subscribing to the cursor position observable.
Test Plan: Browser tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2845
Test Plan: Block read access to column A based on the condition rec.B == 1. Then setting B = 1 in a row makes the cell under A grey.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: paulfitz, dsagal
Differential Revision: https://phab.getgrist.com/D2828
Summary:
A recently added stress test ("deletes documents reasonably quickly" in removedAt.ts) is sporadically failing under postgres. It looks like typeorm's .save() method is in some way unreliable when setting a table with multi-column primary keys, via a ManyToMany relation. This diff replaces the .save() with explicit inserts/deletes.
I modified _repairWorkspaceGuests recently, so thought that change might have been the problem. However under the stress test, failures occur as often in _repairOrgGuests (not changed recently) as in _repairWorkspaceGuests (changed recently).
For reference, see schema diagram at https://grist.quip.com/wWpRAMe058Nl/Home-DB (the table being updated is `group_users`).
Possibly related issue: https://github.com/typeorm/typeorm/issues/4122
Test Plan:
After this change, stress test runs well on postgres locally (no failure 70 iterations); before it would fail on postgres within 3 iterations typically.
Separately: I gave a test that failed a little more time to return, and confirmed it was no slower on average, so I think it was unrelated.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D2848
Summary:
The `_repairWorkspaceGuests` method is slow for workspaces with large numbers of documents. It makes a query that produces a lot of rows. The query itself is tolerable, but TypeORM processing uses enough CPU to be a likely culprit in some production instability. This diff splits the query into two pieces that are logically independent, but which when combined were resulting in the number of rows being the product of the two pieces. Once split, there is also a where clause that can be applied to one of the pieces.
The purpose of the method is to add every user that a document within a workspace is shared with to a "guest" group of the workspace itself. The design of "guest" groups is not ideal, but this diff leaves the design unchanged and is intended only to speed up operation.
Made some small tweaks to the timing of a flakey test, and temporarily recreated the `samples` directory removed in a previous diff (this is currently breaking tests badly on a fresh worker without a `samples` directory lying around)
Test Plan: added test; existing tests pass
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D2844
Summary:
- this is a core search code refactoring
- this diff should fix the js error that was happening when searching across pages.
Test Plan: Tested manually on dev's environment. Tests shows no regression and successfully fixes the js error.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2837
Summary:
This cleans up a few things about SELF_HYPERLINK urls:
* Use `urlId` rather than `docId`.
* Correctly merge personal org subdomain.
* In dev environment, use clearer port number.
Test Plan: updated test
Reviewers: alexmojaki, dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2841
Summary:
Names of private tables and columns were leaking via Code View.
This plugs that leak.
Test Plan: adds test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2840
Summary: Cells will remember their previous state when user pressed the escape key. Grist will offer a way to continue with the draft, by showing notification and a tooltip above the editor.
Test Plan: Browser tests were created
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2822
Summary:
- Sending `ESCAPE` while hovering the options panel was not working
- This is because of the keepExpanded flag which was set to true
- Solution is to set the keepExpanded flag to false, prior to toggling the Menu
Test Plan: - Tested manually
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2836
Test Plan: Set all columns to hidden and reloaded document.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2832
Summary:
We used tslint earlier, and on switching to eslint, some rules were not
transfered. This moves more rules over, for consistent conventions or helpful
warnings.
- Name private members with a leading underscore.
- Prefer interface over a type alias.
- Use consistent spacing around ':' in type annotations.
- Use consistent spacing around braces of code blocks.
- Use semicolons consistently at the ends of statements.
- Use braces around even one-liner blocks, like conditionals and loops.
- Warn about shadowed variables.
Test Plan: Fixed all new warnings. Should be no behavior changes in code.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2831
Test Plan: Created a user attribute under access rules, set the attribute to look up to user.LinkKey.e, confirmed that setting e_ in the URL modified access.
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2824
Summary: Extract out function _onClickHyperlink
Test Plan: Made a table, formatted column as hyperlink, added values including a link to another page in the document, another document, and an external website, clicked on all the links and only the first one didn't open a new tab.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2826
Summary:
This diff implements the search improvement that are mentioned here https://grist.quip.com/j1biAmfGkbzV/Search-Improvements.
CAVEATS:
I've noticed a bit of a usability caveats: the tooltips overlap with the new `search all pages` checkbox, which requires user to move cursor away for a bit in order to be able to click the button.
{F28224}
I've experimented locally with tooltips showing on both sides of the arrows, but it overlaps with the cross icon so could also be an issue. I couldn't think of any clear simple alternative, probably not too big of an issue anyway.
Test Plan: Added new test.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2818
Summary: Editor position wasn't restored on a long list, where the rows haven't been shown yet (the scroll haven't happend yet).
Test Plan: Browser tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2823
Summary:
- The parameter was inadvertently removed by userOverrideParams().
- when passed a function to `urlState().setHref(...)` or `urlState().makeUrl(...)` it is important that the function does not mutate the state that it receives as argument.
Link to the related task: https://gristlabs.getgrist.com/doc/check-ins/p/5#a1.s9.r791.c19
Test Plan: Adds test of the persistence by slightly modifying existing nbrowser/AccessRules2 tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2820
Summary:
For conversions between Choice and ChoiceList, it makes more sense to preserve
the list of choices than to re-parse it from data.
Reported by Anais. Creating Choices from parsing ChoiceList cell values was
particularly poor, resulting in choices like "L,Foo,Bar".
Test Plan: Added a test case
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2819
Summary: Grist document, when reloaded, is able to restore the latest cursor position and the editor state.
Test Plan: Browser test were created.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2808
Summary: Text editor for Integer and Numeric column was showing null or undefined when the underlying value was null.
Test Plan: Browser test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2817
Summary:
Currently, to compute intermediate steps in a bundle, the bundle
is sent to the data engine to process. Then, if the intermediate
steps break a rule, it is reverted. One problem introduced by
checking permissions this late is that the data engine can be
exposed for formulas with python code by users who don't have the
right to change formulas. This diff pre-checks cases that change
formulas.
Test Plan: added a test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2816
Summary:
Auto-complete helps enter correct column names, and when incorrect ones are
entered, we now show an error and prevent saving the rules.
In an unrelated tweak, fix focusing of ACLFormula when clicking into scroll area.
Test Plan: Added a test case for showing invalid columns
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2815
Summary: For row creations and deletions, treat `rec` and `newRec` variables as identical. This simplifies writing a single rule that controls multiple permissions.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2812
Summary:
When scrolling quicly through a column with hyperlinks, null could be passed to
a function that didn't expect it. Added better types would help catch it.
Test Plan: Tested manually
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2813
Summary:
This diff discounts indirect changes for access control purposes. A UserAction that updates a cell A, which in turn causes changes in other dependent cells, will be considered a change to cell A for access control purposes.
The `engine.apply_user_actions` method now returns a `direct` array, with a boolean for each `stored` action, set to `true` if the action is attributed to the user or `false` if it is attributed to the engine. `GranularAccess` ignores actions attributed to the engine when checking for edit rights.
Subtleties:
* Removal of references to a removed row are considered direct changes.
* Doesn't play well with undos as yet. An action that indirectly modifies a cell the user doesn't have rights to may succeed, but it will not be reversible.
Test Plan: added tests, updated tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2806
Summary:
- Adds a new ChoiceList type, and widgets to view and edit it.
- Store in SQLite as a JSON string
- Support conversions between ChoiceList and other types
Test Plan: Added browser tests, and a test for how these values are stored
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2803
Summary: This treats newRec in the same way as rec in access formulas.
Test Plan: updated test for column renames; autocomplete checked manually.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2810
Summary: Saving sort or filter is not permitted in readonly mode. Hence we remove the button. The diff adds a new unit to
Test Plan: Adds test for behaviour to `nbrowser/ReadOnlyMode`. Also adds tests for other Save buttons related to sort & filter that should be disabled.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2804
Summary:
- close sort&filter menu when clicking Save/Revert buttons
- also closes when clicking Apply/Cancel from a nested filter menu
Test Plan:
- updated existing test to match new spec
- added new test to cover new behaviour
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2799
Summary: - Combination of styling of what's in the dropdown and what's in the sort config
Test Plan: adds new nbrowser test
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2798
Summary:
- Clicking quickly on the small save/revert button was caussing the
tooltip to stay around.
- But if user waited a little bit before clicking the save button,
the tooltip was shown, and then properly removed when the button was removed.
- Code was missing propertly handling of disposal before the tooltip
were shown.
Test Plan: Added test case to the projects/tooltip.ts tests
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2797
Summary:
- If you type into the "Everyone" / "Everyone Else" rule, and it stops being
the default rule, there will now be an extra row with a "+" button to add a
new default rule
- Switch to ACE-supported auto-resizing (for better scrollbars handling)
- Tweak ACE padding styles for better-looking scrolling.
Test Plan: Added a test case for the extra "+" button.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2796
Summary:
- Supports auto-completion
- Supports various keyboard interactions to delete/select/copy-paste tokens
- Supports various mouse interactions to select and move tokens.
- Supports undo/redo while editing tokens.
Test Plan: stand-alone fixture and browser test verify the supported behaviors
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2789
Summary:
- New sort and filter button has several states
- Empty / unsaved / saved
- offers small save/revert button when unsaved
- Fix little issue with hanging tooltip when the refElem is disposed.
- The problem was that if you hover the save (or revert) button
and then click the button, it causes the button to disappear,
but the tooltip was staying.
Test Plan: Updated all tests to match the new UI.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal, paulfitz
Differential Revision: https://phab.getgrist.com/D2795
Summary: Select all functionality by clicking the upper left margin corner
Test Plan: Browser test created
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2793
Summary:
When a document has an exception to allow copies,
unset that option on any copies of the document.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2794
Summary:
Exceptional document operations (particularly `system` and `nascent`
operations) should never be denied by a granular access rule.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2792
