Summary: Update _create_syntax_error_code to raise an error with similar arguments to the real arguments it already has, with our modifications.
Test Plan: Updated python unit tests
Reviewers: jarek, dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3040
Summary:
- Anchor links with row of 'new' could be created but weren't parsed or used
correctly. This fixes it.
- Also adds UIRowId type for row IDs which includes the special 'new' row. It's
already been used in places as `number|'new'`, this diff gives it a name usable in app/common
(it doesn't touch another name, RowId, that's been available in app/client).
Test Plan: Added a test assert for anchor links to new row
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D3039
Summary:
See https://grist.quip.com/VKd3ASF99ezD/Outgoing-Webhooks
- 2 new DocApi endpoints: _subscribe and _unsubscribe, not meant to be user friendly or publicly documented. _unsubscribe should be given the response from _subscribe in the body, e.g:
```
$ curl -X POST -H "Authorization: Bearer 8fd4dc59ecb05ab29ae5a183c03101319b8e6ca9" "http://localhost:8080/api/docs/6WYa23FqWxGNe3AR6DLjCJ/tables/Table2/_subscribe" -H "Content-type: application/json" -d '{"url": "https://webhook.site/a916b526-8afc-46e6-aa8f-a625d0d83ec3", "eventTypes": ["add"], "isReadyColumn": "C"}'
{"unsubscribeKey":"3246f158-55b5-4fc7-baa5-093b75ffa86c","triggerId":2,"webhookId":"853b4bfa-9d39-4639-aa33-7d45354903c0"}
$ curl -X POST -H "Authorization: Bearer 8fd4dc59ecb05ab29ae5a183c03101319b8e6ca9" "http://localhost:8080/api/docs/6WYa23FqWxGNe3AR6DLjCJ/tables/Table2/_unsubscribe" -H "Content-type: application/json" -d '{"unsubscribeKey":"3246f158-55b5-4fc7-baa5-093b75ffa86c","triggerId":2,"webhookId":"853b4bfa-9d39-4639-aa33-7d45354903c0"}'
{"success":true}
```
- New DB entity Secret to hold the webhook URL and unsubscribe key
- New document metatable _grist_Triggers subscribes to table changes and points to a secret to use for a webhook
- New file Triggers.ts processes action summaries and uses the two new tables to send webhooks.
- Also went on a bit of a diversion and made a typesafe subclass of TableData for metatables.
I think this is essentially good enough for a first diff, to keep the diffs manageable and to talk about the overall structure. Future diffs can add tests and more robustness using redis etc. After this diff I can also start building the Zapier integration privately.
Test Plan: Tested manually: see curl commands in summary for an example. Payloads can be seen in https://webhook.site/#!/a916b526-8afc-46e6-aa8f-a625d0d83ec3/0b9fe335-33f7-49fe-b90b-2db5ba53382d/1 . Great site for testing webhooks btw.
Reviewers: dsagal, paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3019
Summary: Update LinkingState._makeSrcCellGetter to account for 'new'
Test Plan: Extended test in RightPanelSelectBy.ts
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D3037
Summary:
Used to throw js errors: `Resize must be passed a displayed plot div element.`
Summarizing (or unsummarizing) causes the ChartView view instance to
be replace by a new one in the view layout.
However, the problem is that the old view instance get disposed only
after the new view instance is added to the view layout.
This causes the old view layout to try to resize chart while chart dom
has been removed from the dom (which Plotly does not support).
This diff fixes it by checking the the chart dom elemnt is still in
the dom before making the plotly call to resize the chart.
TODO: It feels weird that the old view instance gets disposed after
the new one get added. Maybe we should check that also.
Test Plan: New test added.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3035
Test Plan: Added a test with lots of time-parsing cases.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D3034
Summary: Constructs a ClientQuery in a similar way to LinkingState to handle lists.
Test Plan: Extended SelectBySummary and SelectByRefList tests.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D3030
Summary:
While switching the destination site in the Duplicate Document dialog, there
were times when it was saveable even though destination workspaces were still
being fetched. This sometimes causes a test failure, with the document getting
saved to a workspace from the previously-selected org.
Test Plan:
Tested manually; reproduced by adding a conditional artificial delay
in _updateWorkspaces helper.
Reviewers: paulfitz, georgegevoian
Reviewed By: paulfitz, georgegevoian
Differential Revision: https://phab.getgrist.com/D3032
Summary:
Also fixes issue with group data options when switching to pie chart.
Issue was that if the group data picker was on, switching to the pie
chart was not hiding it.
Test Plan: Adds more tests.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3028
Summary:
This unsets the `direct` flag for actions emitted when summary tables are updated. That means those actions will be ignored for access control purposes. So if a user has the right to change a source table, the resulting changes to the summary won't result in the overall action bundle being forbidden.
I don't think I've actually seen the use case that inspired this issue being filed. I could imagine perhaps a user forbidden from creating rows globally making permitted updates that could add rows in a summary (and it being desirable to allow that).
Test Plan: added tests
Reviewers: jarek
Reviewed By: jarek
Subscribers: dsagal, alexmojaki, jarek
Differential Revision: https://phab.getgrist.com/D3022
Summary:
This change prevents dragging tokens when the cursor is over the
delete button. Now, trying to drag after mousedown over the delete
button will do nothing if the cursor is released outside the delete
button, and will delete if the cursor is released inside the delete
button.
Test Plan: Tested manually.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D3024
Summary:
A user without SchemaEdit permission was able to reorder pages, since
this changes _grist_Pages, and that table was left under control of
regular access rules. This diff tightens things up, to require
SchemaEdit for all metadata edits. The one remaining exception is
_grist_Attachments, which needs some reworking to play well with
granular access.
Test Plan: extended test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D3025
Summary:
This is a follow up diff for https://phab.getgrist.com/D3021. Y-axis
draggable list used to blink when user changed either one of the x
axis or groupdata column.
This was due to the fact that all of theses axis are stored into the
same array and changing one of them changes the whole array even
though items relative to the y-axis actually were not changing.
This diff addresses this issue by 1) being carefull at not updating
the array of items when the changes do not impact y axis. And 2) by
adding a freeze observable allowing to freeze the draggable list of
y-axis while actions are being treated on the server.
Test Plan:
Catching such bug is hard, and given that it's only look and fill, maybe not worth the time and effort.
Tested manually though.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3023
Summary:
The import dialog now has an option to 'Update existing records',
which when checked will allow for selection of 1 or more fields
to match source and destination tables on.
If all fields match, then the matched record in the
destination table will be merged with the incoming record
from the source table. This means the incoming values will
replace the destination table values, unless the incoming
values are blank.
Additional merge strategies are implemented in the data
engine, but the import dialog only uses one of the
strategies currently. The others can be exposed in the UI
in the future, and tweak the behavior of how source
and destination values should be merged in different contexts,
such as when blank values exist.
Test Plan: Python and browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3020
Summary: This adds a dropdown to the document settings model in staging/dev to set the python engine to Python2 or Python3. The setting is saved in `_grist_DocInfo.documentSettings.engine`.
Test Plan: tested manually for now - separate diff needed to add runsc to jenkins setup and make this testable
Reviewers: dsagal, alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D3014
Summary:
Chart view used to rely on the same view field configuration as used in any other widget.
This diff allows to explicitely select X-AXIS, Y-AXIS and group by column with column picker.
As charts supports several y-axis, we still use a draggable list to arrange them.
Diff also fix doc to the `insertPositions` function.
Test Plan: Updated the relevant test.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3021
Summary: This tests site deletion with and without a plan.
Test Plan: adding tests
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D3017
Summary:
This adds a `site:delete` target to `cli.sh` for deleting sites. Sites should be specified by numeric org id, and for confirmation their name also needs to be given.
All the docs in the site are deleted permanently, and the workspaces, and the site, and the stripe customer (if any).
Test Plan: manual
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D3015
Summary:
This can happen thanks to some badly-behaved extensions (e.g. lingvanex), and
results in errors such as "Cannot redefine property: isInput".
Test Plan:
Tested manually that Grist fails to load in Chrome with Lingvanex
extension, and loads successfully with this change.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3013
Summary:
Moves CSV and XLSX export urls under /download/, and
removes the document title query parameter which is now
retrieved from the backend.
Test Plan: No new tests. Existing tests that verify endpoints still function.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3010
Summary:
Bar chart was a bit broken when there were redundant values on the X axis: the bars’s height maps to the sum of all the corresponding y values, when the data that shows up on hover is only the last one.
It seems that plotly does not support redundant values in the x axis and in all Plotly examples (implementation relies on plotly) x values only have unique values.
This diff, fixes by making sure x axis has unique values. If user actually wants to plot groups, they'll have to use a summary charts.
Test Plan: tested manually
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3011
Summary: We still show up to 5 on regular-width screens.
Test Plan: Browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3008
Summary:
The endpoints for exporting CSV and Excel are now under
/api/docs/:docId/ and are forwarded to a doc worker for export.
The Share Menu has been updated to use the new endpoints.
Test Plan: No new tests. Existing tests that verify endpoints work correctly.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D3007
Summary:
Allows selecting by a reflist in another table. This generalises cursor-linking with a ref column, but now it's filter linking.
Added another case to LinkingState where the source column is a reflist to the target table, filtering by the id column.
Updated convertQueryFromRefs and related functions to handle this since the id column has no column ref. In this case the string 'id' is used instead of a number.
LinkingState also checks if the source value is a reflist and uses that as the list of filter values instead of a single-element list of the cell value.
Indirect linking also works, where the source and target columns both are both references to the same table. This was the plan for a source reflist and target ref column.
I was surprised to see it also works perfectly when both columns are reflists, and it filters rows where there's an intersection!
Adding rows to the target section using the selected source record for default values is iffy. When filtering by row ID, there's no column for defaults, so the new row disappears.
For a source reflist and target ref, the first value of the reflist is the default, which is okayish. When both are reflists, the full source reflist is the default for the target column.
This seems like a bit much but just using the first value seems a bit arbitrary when there's room for all of them?
While doing all this I noticed an unrelated bug which I fixed as I was refactoring. Previously cursor linking based on a reference column did not update the cursor in the link target
when the value of the selected reference cell changed. Now cursor linking uses a floating row model like most other cases to observe the value correctly.
Test Plan: Extended SelectByRefList test and fixture, added previously failing test to RightPanelSelectBy.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D3004
Summary: It is important for linking to be maintained, or user could be gravely misled about material in other sections.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3003
Summary:
The warning about workspace write access would still be shown if
a user picked a workspace they had write access to, and cleared the
Name field in the Save Copy dialog. This fixes the condition for
showing the warning to not show it in this case, and adds a placeholder
to the Name field when it is blank.
Test Plan: Browser test.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D3002
Summary:
- Filters out hidden pages from docModel.allPagesList (used for knowing default page, and for search iteration).
- Filters out hidden pages from TreeModel (uses raw tableData, so has a different way to filter).
Test Plan: WIP
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D2996
Summary:
Access endpoints were supposed to provide display versions of emails,
but in fact only the org endpoint was doing so. This brings the
workspaces and docs endpoints into line, and adds tests.
Full user information is tweaked slightly to return an anonymous
flag only when anonymous. This was already anticipated in the
FullUser type.
Test Plan: extended test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2999
Summary:
- Grist document has a associated "locale" setting that affects how currency is formatted.
- Currency selector for number format.
Test Plan: not done
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D2977
Summary:
This adds `runsc` and `python3` to the grist-server images. For deployments with GRIST_EXPERIMENTAL_PLUGINS=1 (dev + staging but not prod) a hack is added to use `python3` under `runsc` for documents with a special title (`activate-python3-magic` or similar).
This will simplify experiments on behavior of this configuration under realistic conditions.
Hopefully, before landing this, I'll be able to switch to storing a python flag in a document options cell being added by @georgegevoian in a parallel diff, since using the doc title is super hacky :-).
Test Plan: tested manually on worker built locally
Reviewers: dsagal, alexmojaki
Reviewed By: dsagal, alexmojaki
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D2998
Summary: Converted LinkingState from constructor function to class.
Test Plan: no
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2997
Summary:
Previously, using "Change Widget" allowed one to change the underlying table,
but would keep the linking settings. This could allow invalid settings which
would sometimes lead to JS errors. These manifested in production as
"UserError: Query error: n is not a function".
- Unset linking settings in this case, to avoid invalid values.
- In case invalid values are encountered (e.g. saved previously), treat them as
unset, to avoid JS errors.
- If an error does occur, report it with a stack trace.
Also, for testing, added 'selectBy' option to gristUtils helpers for using page-widget-picker.
Test Plan: Added test cases for resetting linking, and for ignoring invalid link settings.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2993
Summary:
Currently actions blocked early because they could modify the
schema (e.g. changing formulas) do not report memo information
(comments in relevant rules). This diff fixes that by using
more of the same code path in the two situations. It also
adds information about what type of action was blocked to
error messages.
Test Plan: extended a test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2995
Summary:
The filter bar used to show in mobile mode while the widget was inactive as illustrated in this screen shot
{F31970}
This diff fixes it.
{F31971}
Test Plan: Manually tested.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D2958
Summary: RecordSets now have new encoding and rendering analogous to Records: `['r', 'Table', [1, 2, 3]]` and `Table[[1, 2, 3]]`.
Test Plan: Added to nbrowser/TypeChange.ts.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2987
Summary:
Currently, if access rules are set to allow edits unconditionally,
and an owner does "View As" a user who is a viewer only, they will
be allowed to make edits. This catches that condition and adds a
test.
Test Plan: added test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D2991
Summary:
- With a format like "DD-MM-YYYY" or "DD MMM YYYY", allow parsing dates
with two digit year or numeric month (like "16-8-21").
- Interpret two-digit years in the same way for moment parsing and for
bootstrap-datepicker.
- For partial inputs (like "8/16"), when a format is present, assume that
provided parts cover the date, then month, then year (even for a format that
starts with year).
Test Plan: Expanded a unittest
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D2985
Summary:
Use 'intersects' query operation when linking against a RefList column, otherwise the rest is the same as linking with a Ref column.
Add RefList columns to Select By options along with Ref columns.
Test Plan: Added new test and fixture similar to SelectBySummary
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2986
Summary:
Pasting data with merged cells from Excel (or from HTML tables with colspan/rowspan),
we used to get "Cannot read property 'displayValue' of undefined".
Fix it by assuming that some cell values may be empty.
Test Plan: Added test case reproduces the failure without the fix, and passes with.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D2990
Summary:
Decoding large actions is a plausible culprit for hogging CPU time for
certain documents. To begin with, log the time taken for this operation,
so that we can tell if it's a problem in practice.
Test Plan: Should not affect any current behaviors
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2989
Summary:
This adds an `updateDomain` billing task that allows editing
the subdomain (and the org name, which is also editable with
the address).
A warning is shown that changing the subdomain will mean that
saved links need updating.
Test Plan: added test
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2988
Summary:
A new section, Other Sites, will now be shown on the All Documents page when:
- A user is on a personal site and has access to other team sites.
- A user is on a public site with view access only.
In addition, a site switcher is now available by clicking
the site name in the top-left section of the UI next to the
Grist logo. It works much like the switcher in the Account
menu.
Test Plan: Browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D2979
Summary: This is a documentation update, and version bump on grist-core.
Test Plan: No code changes.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2982
Summary:
* Remove adjustSession hack, interfering with loading docs under saml.
* Allow the anonymous user to receive an empty list of workspaces for
the merged org.
* Behave better on first page load when org is in path - this used to
fail because of lack of cookie. This is very visible in grist-core,
as a failure to load localhost:8484 on first visit.
* Mark cookie explicitly as SameSite=Lax to remove a warning in firefox.
* Make errorPages available in grist-core.
This changes the default behavior of grist-core to now start off in
anonymous mode, with an explicit sign-in step available. If SAML is not configured,
the sign-in operation will unconditionally sign the user in as a default
user, without any password check or other security. The user email is
taken from GRIST_DEFAULT_EMAIL if set. This is a significant change, but
makes anonymous mode available in grist-core (which is convenient
for testing) and makes behavior with and without SAML much more consistent.
Test Plan: updated test; manual (time to start adding grist-core tests though!)
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2980
Summary:
SAML support had broken due to SameSite changes in browsers. This
makes it work again, and tests it against Auth0 (now owned by Okta).
Logging in and out works. The logged out state is confusing, and may
not be complete. The "Add Account" menu item doesn't work.
But with this, an important part of self-hosting becomes easier.
SAML support works also in grist-core, for site pages, but there
is a glitch on document pages that I'll look into separately.
Test Plan: tested manually
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D2976
