Summary:
This adds `runsc` and `python3` to the grist-server images. For deployments with GRIST_EXPERIMENTAL_PLUGINS=1 (dev + staging but not prod) a hack is added to use `python3` under `runsc` for documents with a special title (`activate-python3-magic` or similar).
This will simplify experiments on behavior of this configuration under realistic conditions.
Hopefully, before landing this, I'll be able to switch to storing a python flag in a document options cell being added by @georgegevoian in a parallel diff, since using the doc title is super hacky :-).
Test Plan: tested manually on worker built locally
Reviewers: dsagal, alexmojaki
Reviewed By: dsagal, alexmojaki
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D2998
Summary: Converted LinkingState from constructor function to class.
Test Plan: no
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2997
Summary:
Previously, using "Change Widget" allowed one to change the underlying table,
but would keep the linking settings. This could allow invalid settings which
would sometimes lead to JS errors. These manifested in production as
"UserError: Query error: n is not a function".
- Unset linking settings in this case, to avoid invalid values.
- In case invalid values are encountered (e.g. saved previously), treat them as
unset, to avoid JS errors.
- If an error does occur, report it with a stack trace.
Also, for testing, added 'selectBy' option to gristUtils helpers for using page-widget-picker.
Test Plan: Added test cases for resetting linking, and for ignoring invalid link settings.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2993
Summary: Uses python unicodedata module to normalise a string and remove combining characters, leaving behind more ascii letters and fewer underscores
Test Plan: Added unit test
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2994
Summary:
Currently actions blocked early because they could modify the
schema (e.g. changing formulas) do not report memo information
(comments in relevant rules). This diff fixes that by using
more of the same code path in the two situations. It also
adds information about what type of action was blocked to
error messages.
Test Plan: extended a test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2995
Summary:
The filter bar used to show in mobile mode while the widget was inactive as illustrated in this screen shot
{F31970}
This diff fixes it.
{F31971}
Test Plan: Manually tested.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2958
Summary:
Our date-guessing logic analyzes text in full looking for date parts.
This diff skip all that work when text is so long that we don't need to
consider it to be a valid date.
This is a quick fix. There are probably many other cases when we don't
need to try hard to parse arbitrary text as dates.
Test Plan: Added a fixture and test case that would trigger the error without the fix.
Reviewers: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2992
Summary: RecordSets now have new encoding and rendering analogous to Records: `['r', 'Table', [1, 2, 3]]` and `Table[[1, 2, 3]]`.
Test Plan: Added to nbrowser/TypeChange.ts.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2987
Summary:
Currently, if access rules are set to allow edits unconditionally,
and an owner does "View As" a user who is a viewer only, they will
be allowed to make edits. This catches that condition and adds a
test.
Test Plan: added test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D2991
Summary:
- With a format like "DD-MM-YYYY" or "DD MMM YYYY", allow parsing dates
with two digit year or numeric month (like "16-8-21").
- Interpret two-digit years in the same way for moment parsing and for
bootstrap-datepicker.
- For partial inputs (like "8/16"), when a format is present, assume that
provided parts cover the date, then month, then year (even for a format that
starts with year).
Test Plan: Expanded a unittest
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D2985
Summary:
Use 'intersects' query operation when linking against a RefList column, otherwise the rest is the same as linking with a Ref column.
Add RefList columns to Select By options along with Ref columns.
Test Plan: Added new test and fixture similar to SelectBySummary
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2986
Summary:
Pasting data with merged cells from Excel (or from HTML tables with colspan/rowspan),
we used to get "Cannot read property 'displayValue' of undefined".
Fix it by assuming that some cell values may be empty.
Test Plan: Added test case reproduces the failure without the fix, and passes with.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2990
Summary:
Decoding large actions is a plausible culprit for hogging CPU time for
certain documents. To begin with, log the time taken for this operation,
so that we can tell if it's a problem in practice.
Test Plan: Should not affect any current behaviors
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2989
Summary:
This adds an `updateDomain` billing task that allows editing
the subdomain (and the org name, which is also editable with
the address).
A warning is shown that changing the subdomain will mean that
saved links need updating.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2988
Summary:
A new section, Other Sites, will now be shown on the All Documents page when:
- A user is on a personal site and has access to other team sites.
- A user is on a public site with view access only.
In addition, a site switcher is now available by clicking
the site name in the top-left section of the UI next to the
Grist logo. It works much like the switcher in the Account
menu.
Test Plan: Browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2979
Summary: This is a documentation update, and version bump on grist-core.
Test Plan: No code changes.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2982
Summary:
* Remove adjustSession hack, interfering with loading docs under saml.
* Allow the anonymous user to receive an empty list of workspaces for
the merged org.
* Behave better on first page load when org is in path - this used to
fail because of lack of cookie. This is very visible in grist-core,
as a failure to load localhost:8484 on first visit.
* Mark cookie explicitly as SameSite=Lax to remove a warning in firefox.
* Make errorPages available in grist-core.
This changes the default behavior of grist-core to now start off in
anonymous mode, with an explicit sign-in step available. If SAML is not configured,
the sign-in operation will unconditionally sign the user in as a default
user, without any password check or other security. The user email is
taken from GRIST_DEFAULT_EMAIL if set. This is a significant change, but
makes anonymous mode available in grist-core (which is convenient
for testing) and makes behavior with and without SAML much more consistent.
Test Plan: updated test; manual (time to start adding grist-core tests though!)
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2980
Summary:
SAML support had broken due to SameSite changes in browsers. This
makes it work again, and tests it against Auth0 (now owned by Okta).
Logging in and out works. The logged out state is confusing, and may
not be complete. The "Add Account" menu item doesn't work.
But with this, an important part of self-hosting becomes easier.
SAML support works also in grist-core, for site pages, but there
is a glitch on document pages that I'll look into separately.
Test Plan: tested manually
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2976
Summary: Adding a custom widget will show the initial page with some information instead of a blank page.
Test Plan: Existing tests
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2978
Summary:
Adds Reference List as a widget type.
Reference List is similar to Choice List: multiple references can be added
to each cell through a similar editor, and the individual references
will always reflect their current value from the referenced table.
Test Plan: Browser tests.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: paulfitz, jarek, alexmojaki, dsagal
Differential Revision: https://phab.getgrist.com/D2959
Summary:
Applies simple data transformations to the existing /data API.
Mimics the Airtable API. Designed in https://grist.quip.com/RZh9AEbPaj8x/Doc-API#FZfACAAZ9a0
Haven't done deletion because it seems like less of a priority and also not fully designed.
Test Plan: Added basic server tests similar to the /data tests. Haven't tested edge cases like bad input.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2974
Summary: Just return a list from _get_col_subset, remove ColumnView class
Test Plan: none
Reviewers: dsagal, georgegevoian
Reviewed By: dsagal, georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D2975
Summary: Updates the summary column type correctly, rebuilds the table model
Test Plan: Added a python unit test, tested manually in browser
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2973
Summary:
Enables the options in the Add New menu when on the templates page
(p/templates).
Test Plan: Browser test.
Reviewers: paulfitz, alexmojaki
Reviewed By: paulfitz, alexmojaki
Subscribers: paulfitz, jarek, alexmojaki
Differential Revision: https://phab.getgrist.com/D2971
Summary: Detached formula uses CONTAINS()
Test Plan: Added to existing unit test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2972
Summary:
Prefix keys of `LinkingState.filterColValues` with `_contains:` when the source column is a ChoiceList or ReferenceList.
This is parsed out to make a boolean `isContainsFilter` which is kept in each value of `QueryRefs.filterTuples` (previously `filterPairs`).
Then when converting back in `convertQueryFromRefs` we construct `Query.contains: {[colId: string]: boolean}`.
Finally `getFilterFunc` uses `Query.contains` to decide what kind of filtering to do.
This is not pretty, but the existing code is already very complex and it was hard to find something that wouldn't require touching loads of code just to make things compile.
Test Plan: Added a new nbrowser test and fixture, tests that selecting a source table by summary tables grouped by a choicelist column, non-list column, and both all filter the correct data.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2940
Summary:
Move all the plugins python code into the main folder with the core code.
Register file importing functions in the same main.py entrypoint as the data engine.
Remove options relating to different entrypoints and code directories. The only remaining plugin-specific option in NSandbox is the import directory/mount, i.e. where files to be parsed are placed.
Test Plan: this
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2965
Summary: Disabling import sources from add menu when a user is not allowed to import.
Test Plan: browser tests
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2970
Summary: Disabling "Add widget to page" on special pages like acl or code
Test Plan: browser tests
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2969
Summary:
Hides the workspace in the breadcrumbs menu if
the doc is unsaved and is not a fork. In practice,
this should usually be when an anonymous user creates a new
document.
Test Plan: Browser tests.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2967
Summary:
Exceptional sessions had lost full read access to documents; this
restores it. Exceptional sessions are used for system actions or
while creating documents.
Test Plan: added test
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2966
Summary: Importing from google drive from home screen (also for anonymous users)
Test Plan: Browser tests
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2943
Summary: UserProfile test fails randomly which is caused by the AccountWidget that loads its data asonchrynously.
Test Plan: Existing tests
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2964
Summary:
When importing from url, user types a url for google spreadsheet,
Grist will switch to Google Drive plugin to allow user to choose file manualy.
Test Plan: Browser tests
Reviewers: paulfitz, dsagal
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2945
Summary: Deletes code which was previously only used by SharedSharing.ts, which was deleted in D2894
Test Plan: no
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2960
Summary:
This adds python2 to the gvisor sandbox image. It can be used instead
of the default python3 by setting PYTHON_VERSION to 2 (or calling run.py with python2).
This is useful for making side-by-side comparisons with code running python3.
Test Plan: manual
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2957
Summary: Improving error messages that get returned from "Import from URL" plugin.
Test Plan: browser tests
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: dsagal, alexmojaki
Differential Revision: https://phab.getgrist.com/D2946
Summary:
get_cell_value wraps RaisedException with CellError to expand the error message for the user.
This is still pretty conceptual, the comments explain some things to think about, but it works and is an improvement.
Test Plan: Updated Python unit tests
Reviewers: dsagal, paulfitz
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2928
Summary:
- Add showGristTour preference, and trigger tour automatically.
- Tour is only triggered for new and anonymous users on a personal org, with
edit permission.
- Automatically open the right panel at tour start.
- Don't show tours on mobile, since that's not ready (popups are cut off
and can't be dismissed)
- Cancel previous tour if a new one is somehow started.
- Remove #repeat- trigger hash tags from the URL when the tour starts.
- Ensure Help Center popup is positioned even when left panel is collapsed.
- Polish up the content of the last two cards in the tour.
Test Plan: Added test case for triggering and opening right panel.
Reviewers: alexmojaki, paulfitz
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2955
Summary:
Adds 'GristDocTour' as a possible value of urlState().docPage
GristDoc checks for this and converts it to a normal view record ID
It also then sets a flag showGristDocTour=true which tells Pages.ts to show the page in the sidebar
Otherwise the page is 'hidden' in the sidebar in the same way it would be if blocked by ACL rules
This all feels very hacky, but I don't know this code well enough to know if there's a better way. Hopefully this behaviour is temporary.
Test Plan: Tested manually, not sure if this is worth an automated test at this stage
Reviewers: paulfitz, dsagal
Reviewed By: paulfitz, dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2953
Summary:
Processing these calls in the client, rather than passing them on
to the backend, means that access rules are more straightforward to
apply.
An unrelated fix is included to filter _grist_ tables when fetched
individually - metadata could leak through this path.
Test Plan: added tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2954
Summary:
Replace Finish button with Previous and an X to close
Add keyboard shortcuts to tour popups
Change last Next button to Finish instead of disabling, can be triggered by Enter key.
Allow closing the tour and reopening in the same place.
Test Plan: only manual, need to confirm desired behaviour
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2950
Summary: This mentions fiddle mode in FullCopies special permission.
Test Plan: visual, existing tests pass
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2952
Summary:
Temporarily adds a client-side check to hide the
Grist Templates org in the Save Copy menu. This will
be removed later once we update getOrgs to optionally
filter orgs that have no workspaces with write access.
Test Plan: Browser tests.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2951
Summary:
Checks for new special columns in GristDocTour: Link_Text, Link_URL, and Link_Icon.
No link is generated if Link_Text is blank or Link_URL cannot be parsed as a URL.
No icon is shown if Link_Icon is not the name of an icon in IconList.ts
Test Plan: Expanded tests, but they now assert things about HTML which may be brittle
Reviewers: georgegevoian, dsagal
Reviewed By: georgegevoian, dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2947
Summary:
Staging tests were using an organization named Test Grist,
while local was using one named test-grist. Both are now
named Test Grist, which should help keep things more consistent
between local and deployment test runs.
The inherited line height of template docs in icon view was
different on Windows, cutting off part of the last line of the
description. The description line height should now be fixed
to a reasonable value.
Test Plan: Manually tested CSS fix on a Windows machine.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2948
Summary:
The old Examples and Templates workspace is now
a page that pulls templates from a new public Grist Templates org.
The All Documents view will pull featured templates from that org, where
featured templates are simply pinned documents in Grist Templates. The
Examples and Templates page will also show the featured templates, as
well as the rest of the available templates organized by category. The
categories are equivalent to workspaces in Grist Templates, and are
generated dynamically.
Test Plan: Browser tests.
Reviewers: paulfitz, dsagal
Reviewed By: paulfitz, dsagal
Subscribers: dsagal, paulfitz, jarek
Differential Revision: https://phab.getgrist.com/D2930
