Summary: Uses python unicodedata module to normalise a string and remove combining characters, leaving behind more ascii letters and fewer underscores
Test Plan: Added unit test
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2994
Summary:
Currently actions blocked early because they could modify the
schema (e.g. changing formulas) do not report memo information
(comments in relevant rules). This diff fixes that by using
more of the same code path in the two situations. It also
adds information about what type of action was blocked to
error messages.
Test Plan: extended a test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2995
Summary:
The filter bar used to show in mobile mode while the widget was inactive as illustrated in this screen shot
{F31970}
This diff fixes it.
{F31971}
Test Plan: Manually tested.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2958
Summary:
Our date-guessing logic analyzes text in full looking for date parts.
This diff skip all that work when text is so long that we don't need to
consider it to be a valid date.
This is a quick fix. There are probably many other cases when we don't
need to try hard to parse arbitrary text as dates.
Test Plan: Added a fixture and test case that would trigger the error without the fix.
Reviewers: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2992
Summary: RecordSets now have new encoding and rendering analogous to Records: `['r', 'Table', [1, 2, 3]]` and `Table[[1, 2, 3]]`.
Test Plan: Added to nbrowser/TypeChange.ts.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2987
Summary:
Currently, if access rules are set to allow edits unconditionally,
and an owner does "View As" a user who is a viewer only, they will
be allowed to make edits. This catches that condition and adds a
test.
Test Plan: added test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D2991
Summary:
- With a format like "DD-MM-YYYY" or "DD MMM YYYY", allow parsing dates
with two digit year or numeric month (like "16-8-21").
- Interpret two-digit years in the same way for moment parsing and for
bootstrap-datepicker.
- For partial inputs (like "8/16"), when a format is present, assume that
provided parts cover the date, then month, then year (even for a format that
starts with year).
Test Plan: Expanded a unittest
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D2985
Summary:
Use 'intersects' query operation when linking against a RefList column, otherwise the rest is the same as linking with a Ref column.
Add RefList columns to Select By options along with Ref columns.
Test Plan: Added new test and fixture similar to SelectBySummary
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2986
Summary:
Pasting data with merged cells from Excel (or from HTML tables with colspan/rowspan),
we used to get "Cannot read property 'displayValue' of undefined".
Fix it by assuming that some cell values may be empty.
Test Plan: Added test case reproduces the failure without the fix, and passes with.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2990
Summary:
Decoding large actions is a plausible culprit for hogging CPU time for
certain documents. To begin with, log the time taken for this operation,
so that we can tell if it's a problem in practice.
Test Plan: Should not affect any current behaviors
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2989
Summary:
This adds an `updateDomain` billing task that allows editing
the subdomain (and the org name, which is also editable with
the address).
A warning is shown that changing the subdomain will mean that
saved links need updating.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2988
Summary:
A new section, Other Sites, will now be shown on the All Documents page when:
- A user is on a personal site and has access to other team sites.
- A user is on a public site with view access only.
In addition, a site switcher is now available by clicking
the site name in the top-left section of the UI next to the
Grist logo. It works much like the switcher in the Account
menu.
Test Plan: Browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2979
Summary: This is a documentation update, and version bump on grist-core.
Test Plan: No code changes.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2982
Summary:
* Remove adjustSession hack, interfering with loading docs under saml.
* Allow the anonymous user to receive an empty list of workspaces for
the merged org.
* Behave better on first page load when org is in path - this used to
fail because of lack of cookie. This is very visible in grist-core,
as a failure to load localhost:8484 on first visit.
* Mark cookie explicitly as SameSite=Lax to remove a warning in firefox.
* Make errorPages available in grist-core.
This changes the default behavior of grist-core to now start off in
anonymous mode, with an explicit sign-in step available. If SAML is not configured,
the sign-in operation will unconditionally sign the user in as a default
user, without any password check or other security. The user email is
taken from GRIST_DEFAULT_EMAIL if set. This is a significant change, but
makes anonymous mode available in grist-core (which is convenient
for testing) and makes behavior with and without SAML much more consistent.
Test Plan: updated test; manual (time to start adding grist-core tests though!)
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2980
Summary:
SAML support had broken due to SameSite changes in browsers. This
makes it work again, and tests it against Auth0 (now owned by Okta).
Logging in and out works. The logged out state is confusing, and may
not be complete. The "Add Account" menu item doesn't work.
But with this, an important part of self-hosting becomes easier.
SAML support works also in grist-core, for site pages, but there
is a glitch on document pages that I'll look into separately.
Test Plan: tested manually
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2976
Summary: Adding a custom widget will show the initial page with some information instead of a blank page.
Test Plan: Existing tests
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2978
Summary:
Adds Reference List as a widget type.
Reference List is similar to Choice List: multiple references can be added
to each cell through a similar editor, and the individual references
will always reflect their current value from the referenced table.
Test Plan: Browser tests.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: paulfitz, jarek, alexmojaki, dsagal
Differential Revision: https://phab.getgrist.com/D2959
Summary:
Applies simple data transformations to the existing /data API.
Mimics the Airtable API. Designed in https://grist.quip.com/RZh9AEbPaj8x/Doc-API#FZfACAAZ9a0
Haven't done deletion because it seems like less of a priority and also not fully designed.
Test Plan: Added basic server tests similar to the /data tests. Haven't tested edge cases like bad input.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2974
Summary: Just return a list from _get_col_subset, remove ColumnView class
Test Plan: none
Reviewers: dsagal, georgegevoian
Reviewed By: dsagal, georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D2975
Summary: Updates the summary column type correctly, rebuilds the table model
Test Plan: Added a python unit test, tested manually in browser
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2973
Summary:
Enables the options in the Add New menu when on the templates page
(p/templates).
Test Plan: Browser test.
Reviewers: paulfitz, alexmojaki
Reviewed By: paulfitz, alexmojaki
Subscribers: paulfitz, jarek, alexmojaki
Differential Revision: https://phab.getgrist.com/D2971
Summary: Detached formula uses CONTAINS()
Test Plan: Added to existing unit test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2972
Summary:
Prefix keys of `LinkingState.filterColValues` with `_contains:` when the source column is a ChoiceList or ReferenceList.
This is parsed out to make a boolean `isContainsFilter` which is kept in each value of `QueryRefs.filterTuples` (previously `filterPairs`).
Then when converting back in `convertQueryFromRefs` we construct `Query.contains: {[colId: string]: boolean}`.
Finally `getFilterFunc` uses `Query.contains` to decide what kind of filtering to do.
This is not pretty, but the existing code is already very complex and it was hard to find something that wouldn't require touching loads of code just to make things compile.
Test Plan: Added a new nbrowser test and fixture, tests that selecting a source table by summary tables grouped by a choicelist column, non-list column, and both all filter the correct data.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2940
Summary:
Move all the plugins python code into the main folder with the core code.
Register file importing functions in the same main.py entrypoint as the data engine.
Remove options relating to different entrypoints and code directories. The only remaining plugin-specific option in NSandbox is the import directory/mount, i.e. where files to be parsed are placed.
Test Plan: this
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2965
Summary: Disabling import sources from add menu when a user is not allowed to import.
Test Plan: browser tests
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2970
Summary: Disabling "Add widget to page" on special pages like acl or code
Test Plan: browser tests
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2969
Summary:
Hides the workspace in the breadcrumbs menu if
the doc is unsaved and is not a fork. In practice,
this should usually be when an anonymous user creates a new
document.
Test Plan: Browser tests.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2967
Summary:
Exceptional sessions had lost full read access to documents; this
restores it. Exceptional sessions are used for system actions or
while creating documents.
Test Plan: added test
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2966
Summary: Importing from google drive from home screen (also for anonymous users)
Test Plan: Browser tests
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2943
Summary: UserProfile test fails randomly which is caused by the AccountWidget that loads its data asonchrynously.
Test Plan: Existing tests
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2964
Summary:
When importing from url, user types a url for google spreadsheet,
Grist will switch to Google Drive plugin to allow user to choose file manualy.
Test Plan: Browser tests
Reviewers: paulfitz, dsagal
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2945
Summary: Deletes code which was previously only used by SharedSharing.ts, which was deleted in D2894
Test Plan: no
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2960
Summary:
This adds python2 to the gvisor sandbox image. It can be used instead
of the default python3 by setting PYTHON_VERSION to 2 (or calling run.py with python2).
This is useful for making side-by-side comparisons with code running python3.
Test Plan: manual
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2957
Summary: Improving error messages that get returned from "Import from URL" plugin.
Test Plan: browser tests
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: dsagal, alexmojaki
Differential Revision: https://phab.getgrist.com/D2946
Summary:
get_cell_value wraps RaisedException with CellError to expand the error message for the user.
This is still pretty conceptual, the comments explain some things to think about, but it works and is an improvement.
Test Plan: Updated Python unit tests
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2928
Summary:
- Add showGristTour preference, and trigger tour automatically.
- Tour is only triggered for new and anonymous users on a personal org, with
edit permission.
- Automatically open the right panel at tour start.
- Don't show tours on mobile, since that's not ready (popups are cut off
and can't be dismissed)
- Cancel previous tour if a new one is somehow started.
- Remove #repeat- trigger hash tags from the URL when the tour starts.
- Ensure Help Center popup is positioned even when left panel is collapsed.
- Polish up the content of the last two cards in the tour.
Test Plan: Added test case for triggering and opening right panel.
Reviewers: alexmojaki, paulfitz
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2955
Summary:
Adds 'GristDocTour' as a possible value of urlState().docPage
GristDoc checks for this and converts it to a normal view record ID
It also then sets a flag showGristDocTour=true which tells Pages.ts to show the page in the sidebar
Otherwise the page is 'hidden' in the sidebar in the same way it would be if blocked by ACL rules
This all feels very hacky, but I don't know this code well enough to know if there's a better way. Hopefully this behaviour is temporary.
Test Plan: Tested manually, not sure if this is worth an automated test at this stage
Reviewers: paulfitz, dsagal
Reviewed By: paulfitz, dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2953
Summary:
Processing these calls in the client, rather than passing them on
to the backend, means that access rules are more straightforward to
apply.
An unrelated fix is included to filter _grist_ tables when fetched
individually - metadata could leak through this path.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2954
Summary:
Replace Finish button with Previous and an X to close
Add keyboard shortcuts to tour popups
Change last Next button to Finish instead of disabling, can be triggered by Enter key.
Allow closing the tour and reopening in the same place.
Test Plan: only manual, need to confirm desired behaviour
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2950
Summary: This mentions fiddle mode in FullCopies special permission.
Test Plan: visual, existing tests pass
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2952
Summary:
Temporarily adds a client-side check to hide the
Grist Templates org in the Save Copy menu. This will
be removed later once we update getOrgs to optionally
filter orgs that have no workspaces with write access.
Test Plan: Browser tests.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2951
Summary:
Checks for new special columns in GristDocTour: Link_Text, Link_URL, and Link_Icon.
No link is generated if Link_Text is blank or Link_URL cannot be parsed as a URL.
No icon is shown if Link_Icon is not the name of an icon in IconList.ts
Test Plan: Expanded tests, but they now assert things about HTML which may be brittle
Reviewers: georgegevoian, dsagal
Reviewed By: georgegevoian, dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2947
Summary:
Staging tests were using an organization named Test Grist,
while local was using one named test-grist. Both are now
named Test Grist, which should help keep things more consistent
between local and deployment test runs.
The inherited line height of template docs in icon view was
different on Windows, cutting off part of the last line of the
description. The description line height should now be fixed
to a reasonable value.
Test Plan: Manually tested CSS fix on a Windows machine.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2948
Summary:
The old Examples and Templates workspace is now
a page that pulls templates from a new public Grist Templates org.
The All Documents view will pull featured templates from that org, where
featured templates are simply pinned documents in Grist Templates. The
Examples and Templates page will also show the featured templates, as
well as the rest of the available templates organized by category. The
categories are equivalent to workspaces in Grist Templates, and are
generated dynamically.
Test Plan: Browser tests.
Reviewers: paulfitz, dsagal
Reviewed By: paulfitz, dsagal
Subscribers: dsagal, paulfitz, jarek
Differential Revision: https://phab.getgrist.com/D2930
Summary:
* Moves essential plugins to grist-core, so that basic imports (e.g. csv) work.
* Adds support for a `GRIST_SANDBOX_FLAVOR` flag that can systematically override how the data engine is run.
- `GRIST_SANDBOX_FLAVOR=pynbox` is "classic" nacl-based sandbox.
- `GRIST_SANDBOX_FLAVOR=docker` runs engines in individual docker containers. It requires an image specified in `sandbox/docker` (alternative images can be named with `GRIST_SANDBOX` flag - need to contain python and engine requirements). It is a simple reference implementation for sandboxing.
- `GRIST_SANDBOX_FLAVOR=unsandboxed` runs whatever local version of python is specified by a `GRIST_SANDBOX` flag directly, with no sandboxing. Engine requirements must be installed, so an absolute path to a python executable in a virtualenv is easiest to manage.
- `GRIST_SANDBOX_FLAVOR=gvisor` runs the data engine via gvisor's runsc. Experimental, with implementation not included in grist-core. Since gvisor runs on Linux only, this flavor supports wrapping the sandboxes in a single shared docker container.
* Tweaks some recent express query parameter code to work in grist-core, which has a slightly different version of express (smoke test doesn't catch this since in Jenkins core is built within a workspace that has node_modules, and wires get crossed - in a dev environment the problem on master can be seen by doing `buildtools/build_core.sh /tmp/any_path_outside_grist`).
The new sandbox options do not have tests yet, nor does this they change the behavior of grist servers today. They are there to clean up and consolidate a collection of patches I've been using that were getting cumbersome, and make it easier to run experiments.
I haven't looked closely at imports beyond core.
Test Plan: tested manually against regular grist and grist-core, including imports
Reviewers: alexmojaki, dsagal
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2942
Summary:
Extracts code from showExampleCard into a generic function which is reused for document tours.
It handles reading and writing to user preferences for automatic showing and explicitly reopening.
Test Plan:
Manually tested that it automatically shows a tour just once and clicking to reopen works.
There's not much new functionality so there's little that needs testing. This is an initial version that's mostly internal and is likely to be polished for users in the future.
If I should still add tests, I'd like confirmation that the current behaviour is as desired.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2944
Summary:
Like the welcome tour, a special URL hash triggers startDocTour which uses data from a table GristDocTour to construct the appropriate popups.
This is the basic version described in https://grist.quip.com/sN2RAHI2dchm/Document-tours
Test Plan:
Added a new nbrowser test which tests the data produced by makeDocTour. The general behaviour of the UI and popups has hardly changed so existing tests cover that well enough.
The new test uses a new fixture document which you can open to easily experience the tour.
Error cases where there's no valid document tour are not tested because that behaviour is likely to change significantly and this feature is still quite 'private'.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: jarek, dsagal
Differential Revision: https://phab.getgrist.com/D2938
