The `Origin` header is produced by `getTransitiveHeaders` but we don't
need it here, as this is only for an internal request where no
cross-origin attacks are possible.
Summary:
When one of 2 widget was collapsed, the resulting widget can become a root section. Then,
when a page was duplicated, the layout was duplicated incorrectly (with wrong collapsed
section). This resulted in a bug, when the root section was deleted, as it was the last
section in the saved layout, but not the last section on the visible layout.
Test Plan: Added 2 tests
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4227
Summary:
New endpoint `/api/version` that returns latest version of stable docker image in format:
```
{"latestVersion":"1.1.12","
updatedAt":"2024-03-06T06:28:25.752337Z","
isCritical":false,
"updateURL":"https://hub.docker.com/r/gristlabs/grist"
}
```
It connects to docker hub API and reads the version from the tag lists endpoint.
Stores telemetry passed from the client such us: current version, deployment type, installationId and others.
Test Plan: Added new test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4220
* Shutdown Doc worker when it is not considered as available in Redis
* Use isAffirmative for GRIST_MANAGED_WORKERS
* Upgrade Sinon for the tests
* Run Smoke test with pages in English
* Add logic in /status endpoint
Summary:
Front-end code can now test if emails are enabled
and hide some parts of UI based on it.
Test Plan:
Only secondery text was hidden on add users dialog.
Tested manually.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4221
Summary:
Some WS-related code was touched in a recent PR to grist-core. This extends
those changes to the rest of the codebase so that builds work again.
Test Plan: N/A
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4224
Summary: Summary tables have restrictions on which columns can be renamed.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D4222
The motivation for supporting an alternative to WebSockets is that while all browsers supported by Grist offer native WebSocket support, some networking environments do not allow WebSocket traffic.
Engine.IO is used as the underlying implementation of HTTP long polling. The Grist client will first attempt a regular WebSocket connection, using the same protocol and endpoints as before, but fall back to long polling using Engine.IO if the WebSocket connection fails.
Include these changes:
- CORS websocket requests are now rejected as a stronger security measure. This shouldn’t affect anything in practice; but previously it could be possible to make unauthenticated websocket requests from another origin.
- GRIST_HOST variable no longer affects CORS responses (also should not affect anything in practice, as it wasn't serving a useful purpose)
Summary:
When access rules are in effect on a document, non-owners currently
don't have access to snapshots. Previously when the document history
tab is opened in this situation, an error toast would appear, along
with a small message that was hard to see in dark mode. This change
removes the toast and improves the message somewhat.
Test Plan: updated test
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4218
Summary:
- Add InstallAdmin class to identify users who can manage Grist installation.
This is overridable by different Grist flavors (e.g. different in SaaS).
It generalizes previous logic used to decide who can control Activation
settings (e.g. enable telemetry).
- Implement a basic Admin Panel at /admin, and move items previously in the
"Support Grist" page into the "Support Grist" section of the Admin Panel.
- Replace "Support Grist" menu items with "Admin Panel" and show only to admins.
- Add "Support Grist" links to Github sponsorship to user-account menu.
- Add "Support Grist" button to top-bar, which
- for admins, replaces the previous "Contribute" button and reopens the "Support Grist / opt-in to telemetry" nudge (unchanged)
- for everyone else, links to Github sponsorship
- in either case, user can dismiss it.
Test Plan: Shuffled some test cases between Support Grist and the new Admin Panel, and added some new cases.
Reviewers: jarek, paulfitz
Reviewed By: jarek, paulfitz
Differential Revision: https://phab.getgrist.com/D4194
Summary:
- Forms now have a reset button.
- Choice and Reference fields in forms now have an improved select menu.
- Formula and attachments column types are no longer mappable or visible in forms.
- Fields in a form widget are now removed if their column is deleted.
- The preview button in a published form widget has been replaced with a view button. It now opens the published form in a new tab.
- A new share menu for published form widgets, with options to copy a link or embed code.
- Forms can now have multiple sections.
- Form widgets now indicate when publishing is unavailable (e.g. in forks or unsaved documents).
- General improvements to form styling.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4203
Summary: Adds support for importing .dsv files (an April Fools 2024 easter egg), and options for exporting .dsv and .tsv files from the Share menu.
Test Plan: Browser and server tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4210
Summary:
Drag-and-drop was interfering with text selection in inputs for form
labels and paragraphs.
Test Plan: Manual.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D4209
Summary: Using the lookup shortcut in the add column menu to find a max or min value is a buggy experience. MAX and MIN returns 0 for empty collections which can be interpreted as 1970.1.1 in date or date time columns. It is better to just return None (empty cell) when there are no records.
Test Plan: Updated
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4206
Summary:
Site access was being used to show or hide form widget buttons. This meant
that users granted owner access to a document couldn't see buttons to
publish/unpublish or share a form. Now, document access is used instead.
Test Plan: Manual.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4207
This check should be unnecessary for stores with strong consistency guarantees (virtually everywhere now).
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary: Empty doc tours are now skipped. Before, an error was shown instead.
Test Plan: Browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4204
Summary:
- UI now trims column rules before saving.
- When rules are loaded, bits that aren't applicable to a resource get ignored.
This should fix the incorrect behavior in existing docs without a migration.
Test Plan:
- Added test of UI, that it now sends trimmed rules
- Added a unitteset of new trimPermissions() function
- Add test of fixed interpretation of existing rules: now only permission bits
applicable to a resource get respected. I.e. create/delete/schemaEdit are
ignored in column rules, and schemaEdit is also ignored in table rules.
- Note that DuplicateTest was affected: updated on the assumption that
schemaEdit still can't actually apply at a table level.
Reviewers: georgegevoian, paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4205
Summary: Adds CSS to support dark mode in Swagger UI.
Test Plan: Manual.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4199
This is a start at a page for diagnosing problems while setting up Grist. Starting to add some diagnostics based on feedback in github issues. We should make Grist installation easier! But when there is a problem it should be easier to diagnose than it is now, and this may help. The page is ugly and doesn't have many diagnostics yet, but we can iterate.
Visit `/boot` on a Grist server for tips on how to use this feature.
Summary: Showing configuration screen when widget is not mapped
Test Plan: New test added
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4192
Summary:
- Rename "Add Default Rule" to "Add Table-wide Rule", and enable always.
- Rename "All Other" to "All".
Test Plan: Updated test; new behavior (for when menu option is newly enabled) only tested manually.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4195
Summary:
In some environments, URLs produced by FormAPI were missing an org, making them
ambiguous. This ensures that an org is always included in the path.
Test Plan: Manual.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4200
Hyphens are typically used by MinIO in version ids (these are uuids).
We need to escape them as they are not included in the path to load a document
after bcb9740
Summary:
There has been inconsistency in using display email vs normalized email, which
ends up creating some duplication in downstream analyses (e.g. the same user
showing up twice with different capitalization).
1. Add UserProfile.loginEmail field with normalized email to prefer, when set, over the inconsistently used UserProfile.email.
2. In one place where it's not available, normalize the display email manually.
3. Clean up some code in Client.ts.
Unrelated tweak to API Console to be clear when a URL parameter wasn't found (rather than show whatever happens to be the first value).
Several test robustness improvements:
- Misplaced parenthesis in gristWebDriverUtils has been causing optTimeout argument to be ignored in tests, and treated always as indefinite.
- Attempt to fix SortMenu test by ignoring (retrying with logging) errors in waitForServer, which include "script timeout" errors that come from a non-configurable selenium or chromedriver timeout.
- Attempt to improve onNewTab() helper, which plays a role in failing Billing tests.
Test Plan: Tested manually the capitalization of logged emails. Counting on existing tests to catch issues.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4188
Summary:
Fixes misc. bugs with forms, updates Grist URLs on static form pages to link
to the new forms marketing page, and adds a forms announcement popup that's
shown next to the Add New button within a document.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4185
Summary: The new events capture usage of forms, widgets, access rules, and onboarding tours and tips.
Test Plan: Manual.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4189
Summary:
few things is going here:
1. Added comand that can be called with intent to show behavioral popup somewhere else. I've added it to trigger showing reference popup from new colum menu, despite popup existing in FieldBuilder
2. Command for showing right panel get an argument to switch for choosen tab right after showing.
Test Plan: two test added - one to check if right side menu is shown after chosing ref or refList, and second to check if popup is shown.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4138
Summary:
If the longer OpenAI model exceeds the OpenAPI context length, we now perform another retry with a
shorter variant of the formula prompt. The shorter prompt excludes non-referenced tables and lookup
method definitions, which should help reduce token usage in documents with larger schemas.
Test Plan: Server test.
Reviewers: JakubSerafin
Reviewed By: JakubSerafin
Subscribers: JakubSerafin
Differential Revision: https://phab.getgrist.com/D4184
Summary:
A negative value was being used for the row id of the
"Add New" row in some cases. This broke selection when a
table was empty, which consequently caused other bugs.
Test Plan: Browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4180
Summary:
On narrow screens, there wasn't enough room in the top bar to see the
text you were typing into the search input. To make more room, items to
the left of the search input are now hidden while search is open; the document
title is hidden on narrow screens, and the undo and redo buttons are always
hidden.
Test Plan: Manual.
Reviewers: JakubSerafin
Reviewed By: JakubSerafin
Subscribers: JakubSerafin
Differential Revision: https://phab.getgrist.com/D4187
Summary:
Adds a new test for formatting and fix several related bugs it uncovered:
1. When editing a number with "," decimal separator, ensure it opens in
the editor with "," (rather than ".", the original bug motivating this).
2. When guessing number format, set maxDecimals when it's needed
(otherwise, e.g. "$1.234", or "4.5%" weren't guessed as numeric)
3. When guessing number format, ignore whitespace when deciding if
guessed format is correct (otherwise percents can't be guessed in
locales which add "%" with a non-breaking space before it).
Test Plan: Added a test case that exercises all fixed behaviors.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4177
Summary:
Login (and other) middleware was included in the public form URL by mistake,
forcing logins on forms hosted on non-personal sites.
Test Plan: Browser test.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4181
Without this optimization, we fetched loads of entries from the database, which led to database and nodejs overloads.
We could go further, this is a modest patch towards better performance.
We use two queries: one fetches the workspaces, the second the organization that the workspace belongs to.
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary:
Creator panel was showing a header with tabs on Raw Data page. This
was broken by forms feature lately.
Test Plan: Added
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4179
Summary:
Improvements
- Widget and column descriptions are now copied when duplicating a table.
- A Grist Plugin API command to open a Record Card is now available.
- New Card widgets set initial settings based on those used by their table's
Record Card.
Fixes
- Opening a reference in a Record Card from a Raw Data popup now opens
the correct reference.
Test Plan: Browser and python tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4164
Summary:
Changes to building and serving:
- Remove unpkg dependencies, add npm module for swagger-ui-dist instead.
- Move apiconsole JS logic into core/app/client/apiconsole.ts, and use TypeScript.
- Add symlinks to swagger in static/ and core/static/.
- Refactor loadScript, and add loadCssFile; use these to load swagger-ui resources.
Changes to console itself:
- Support docId, workspaceId, orgId URL parameters. When present, the matching
value in dropdowns is moved to the front and marked as "(Current)".
- Fix the ordering of example values, particularly for workspaces.
- Remove unwanted example values.
- Hide confusing "Authorize" button.
- Hide API keys, and rely consistently on cookies for executing API calls.
Integration into Grist:
- Added a button to Document Settings, just under document ID in "API".
- The button opens a separate page, passing in org, workspace, and doc info for the current doc.
Test Plan: Only tested manually, no automated tests yet.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4173
Summary:
This offers better handling for accented and non-English characters,
e.g. "Événements" becomes "Evenements" (rather than "vnements") and
"таблиця" becomes "tablicya" rather than an empty string.
Test Plan:
Added a test case. Existing documents will auto-redirect to
newly-generated names. In cases where it's different, that's probably for the
best.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4176
Summary:
Takes some numbers embedded in strings and moves them to the context, so
they can be aggregated and trends inspected more easily.
Test Plan: tested manually by looking at logs
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4175
Summary:
- Adding little green asterisk at the end of field title.
- Fixing bug on columns component. Adding paragraph as a column and then selecting it was throwing error in the RightPanel
- Fixing boolean column bug in the editor
- Adding (--Choose--) placeholder for dropdowns
- Fixing columns logic: Dragging and dropping columns can unexpectedly add more columns.
- Added favicon and default page title
- Added svg to sync file for electron.
Test Plan: Updated
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4172
Summary:
- Updates styling of form submitted page.
- Tweaks styling of checkboxes, labels, and questions on form page.
- Adds new form 404 page.
- Adds checkbox to not show warning again when publishing or un-publishing a form.
- Excludes formula, hidden, and attachment columns in submitted form data.
- Adds placeholder text to form configuration inputs.
- Improves dark mode styling in Form widget.
- Updates default title and description of new forms.
- Updates styling of Form widget buttons.
- Fixes form success text input handling.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4170
Summary:
Enhances sign-up telemetry with login and verification method metadata, and
adds UTM parameters to SendGrid invite email links and Grist document links.
Test Plan: Server and manual.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4169
Summary:
This moves the `formUrl` logic to `encodeUrl`, which is more
aware of how the URL is constructed than UserAPI. UserAPI can
only reliably construct API URLs.
Test Plan: extended tests
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4171
Summary:
- Open all links in a new tab
- Excluding not filled columns (to fix trigger formulas)
- Fixed Ref/RefList submission
- Removing redundant type definitions for Box
- Adding header menu item
- Default empty values in select control
Test Plan: Updated
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4166
* Fix server crash when client passes malformed JSON
* Take remarks into account
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary:
The first time a worker opens a document, it will now check if it has any shares the home db needs to be aware of. If so, they will be added. This is important for documents uploaded/copied/forked/replaced, so that their shares work out of the box.
In future, may want some UI to give user control of whether shares are activated after upload/copy/fork/replace.
It seems tricky currently to know if a document is being opened for the first time. As a proxy, I check whether usage information has been calculated and saved to the db, since I can determine that without adding another db query. It is safe to synchronize shares more than necessary.
This leaves two gaps:
* If a document is created/uploaded/copied/forked/replaced and no attempt is made to access it prior to using a share, then that share won't actually be available. Not a problem currently I think, since how would a user have determined the share key. But in future it would be good to also do a sync after creation/upload/copy/fork/replacement/...
* On document replacement, usage info is reset but not absolutely immediately. So in principle shares could fail to be created on first load of the replacement. Usage info reset could be tweaked to give a guarantee here, but also fixing the first point would resolve this second point too.
Test Plan: copy test added
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4165
Summary:
This corresponding change in HelpCenter will report this event:
https://github.com/gristlabs/grist-help/pull/303
Test Plan: Tested locally that the telemetry event is received by the server and logged.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4168
Summary:
The "Create a form" button in section menu wasn't hidden
behind a feature flag.
Test Plan: Manual
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4167
Summary:
Forms improvements and following new design
- New headers
- New UI
- New right panel options
Test Plan: Tests updated
Reviewers: georgegevoian, dsagal
Reviewed By: georgegevoian
Subscribers: dsagal, paulfitz
Differential Revision: https://phab.getgrist.com/D4158
Summary:
Fixes a recent regression that would cause a record to be erroneously filtered
out whenever it was updated from a linked view.
Test Plan: Browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4163
Summary:
Adds initial implementation of form publishing, built upon WYSIWYS shares.
A simple UI for publishing and unpublishing forms is included.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: paulfitz, jarek
Differential Revision: https://phab.getgrist.com/D4154
Summary:
- Move CellValue documentation to CellValue and add an example for each type.
- Link to CellValue from places that mention it.
- Update deprecated [[LINK]] syntax to a supported {@link} one, in a way that
does not change generated documentation.
- Also fix auto-pick-ports script used in tests (which had a bug causing occasional test failures), and add a test for it.
Test Plan: No code changes for documentation changes. New test for auto-pick-ports.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4162
Summary:
When linking using a Reference List column, there may be multiple source
records that show the same target record. With this change, we remember those
(rather than just pick one that shows the target record).
Test Plan: Added a browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4140
Summary:
While adding `FetchSelectedOptions` in 4e67c679b2, I accidentally made `viewApi` become an undocumented variable, which is fixed here.
While I was at it, I also fixed other errors emitted by `./build-plugin-api.sh`.
Test Plan: Built changes in https://github.com/gristlabs/grist-help/pull/297
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4155
Summary:
This gives a mechanism for controlling access control within a document that is distinct from (though implemented with the same machinery as) granular access rules.
It was hard to find a good way to insert this that didn't dissolve in a soup of complications, so here's what I went with:
* When reading rules, if there are shares, extra rules are added.
* If there are shares, all rules are made conditional on a "ShareRef" user property.
* "ShareRef" is null when a doc is accessed in normal way, and the row id of a share when accessed via a share.
There's no UI for controlling shares (George is working on it for forms), but you can do it by editing a `_grist_Shares` table in a document. Suppose you make a fresh document with a single page/table/widget, then to create an empty share you can do:
```
gristDocPageModel.gristDoc.get().docData.sendAction(['AddRecord', '_grist_Shares', null, {linkId: 'xyz', options: '{"publish": true}'}])
```
If you look at the home db now there should be something in the `shares` table:
```
$ sqlite3 -table landing.db "select * from shares"
+----+------------------------+------------------------+--------------+---------+
| id | key | doc_id | link_id | options |
+----+------------------------+------------------------+--------------+---------+
| 1 | gSL4g38PsyautLHnjmXh2K | 4qYuace1xP2CTcPunFdtan | xyz | ... |
+----+------------------------+------------------------+--------------+---------+
```
If you take the key from that (gSL4g38PsyautLHnjmXh2K in this case) and replace the document's urlId in its URL with `s.<key>` (in this case `s.gSL4g38PsyautLHnjmXh2K` then you can use the regular document landing page (it will be quite blank initially) or API endpoint via the share.
E.g. for me `http://localhost:8080/o/docs/s0gSL4g38PsyautLHnjmXh2K/share-inter-3` accesses the doc.
To actually share some material - useful commands:
```
gristDocPageModel.gristDoc.get().docData.getMetaTable('_grist_Views_section').getRecords()
gristDocPageModel.gristDoc.get().docData.sendAction(['UpdateRecord', '_grist_Views_section', 1, {shareOptions: '{"publish": true, "form": true}'}])
gristDocPageModel.gristDoc.get().docData.getMetaTable('_grist_Pages').getRecords()
gristDocPageModel.gristDoc.get().docData.sendAction(['UpdateRecord', '_grist_Pages', 1, {shareRef: 1}])
```
For a share to be effective, at least one page needs to have its shareRef set to the rowId of the share, and at least one widget on one of those pages needs to have its shareOptions set to {"publish": "true", "form": "true"} (meaning turn on sharing, and include form sharing), and the share itself needs {"publish": true} on its options.
I think special shares are kind of incompatible with public sharing, since by their nature (allowing access to all endpoints) they easily expose the docId, and changing that would be hard.
Test Plan: tests added
Reviewers: dsagal, georgegevoian
Reviewed By: dsagal, georgegevoian
Subscribers: jarek, dsagal
Differential Revision: https://phab.getgrist.com/D4144
The error can often be fixed by just reloading the document with no need
to worry the document owners
For example, when the error message is: "interrupted by reconnect"
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary:
This removes checking for full access in `onRecord/onRecords` when `includeColumns` is a non-default value. The check had two problems:
1. It relied on the access level being present in the URL query parameters, which doesn't work if the page has redirected. See the discussion in https://grist.slack.com/archives/C0234CPPXPA/p1702576602615509. There seems to be no way to reliably and synchronously check the access level.
2. Calling `onRecords` before `ready` and forgetting to handle an error from the access check meant that `ready` wouldn't be called, so Grist couldn't request the correct access level from the user. I made this mistake and it seems like a nasty footgun.
Ultimately this has no effect on security, as an error will still be raised, but in a place where the widget developer can't catch it. They'll still see an error message in the console, and they can still check the access level reliably using `onOptions`, so I think this is OK.
Test Plan: Updated nbrowser test
Reviewers: georgegevoian, paulfitz
Reviewed By: georgegevoian, paulfitz
Differential Revision: https://phab.getgrist.com/D4145
Summary: Adds a custom interactive Swagger API console at `/apiconsole`. For now, this isn't visibly linked anywhere.
Test Plan: Manual, this is still an experimental and private feature. The idea is to merge this soon so that we have a chance to try it out in production.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4151
Summary:
This moves some more tests to core that would be useful for ANCT,
which had been stuck in grist-saas due to some entanglements with
sendgrid and billing. For sendgrid, I've moved around just enough
material to permit the tests to run mostly unchanged. Ideally
the interface to a notification system would be generalized, but
that's a bigger project.
Test Plan:
checked that tests are likely to run as expected
in core using preview laid out by ./buildtools/build_core.sh
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4149
Test Plan: Tested manually that strange-looking shortcut line for "Detach active editor" is gone.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4148
Summary:
A new widget type Forms. For now hidden behind GRIST_EXPERIMENTAL_PLUGINS().
This diff contains all the core moving parts as a serves as a base to extend this functionality
further.
Test Plan: New test added
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4130
Summary:
If linking state changes multiple times frequently the code that simulates async operation is
wrongly debounced, which causes inverted order of execution. This fix makes sure that only the last
call to filter function is used.
Test Plan: Adding new test
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D4139
Summary:
With both panels expanded and a narrow viewport, the tables on the
Raw Data page would visibly overflow. This improves things so that
overflow is handled more gracefully.
Test Plan: Manual.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4147
Summary:
When a widget `A` is selected by a widget `B` so that `A` is filtered, adding a new row to `A` uses the values in the selected row of `B` and the columns relevant to the linking as default values for the new row. This ensures that the new row matches the current linking filter and remains visible. However this would previously cause a sandbox error when one of the linking columns was a formula column, which doesn't allow setting values. This diff ignores formula columns when picking default values.
Since the value of the formula column in the new row typically won't match the linking filter, extra measures are needed to avoid the new row immediately disappearing. Regular filters already have a mechanism for this, but I didn't manage to extend it to also work for linking. Thanks @dsagal for creating `UnionRowSource` (originally in D4017) which is now used as the solution for temporarily exempting rows from both kinds of filtering.
While testing, I also came across another bug in linking summary tables that caused incorrect filtering, which I fixed with some changes to `DynamicQuerySet`.
Test Plan: Extended an nbrowser test, which both tests for the main change as well as the secondary bugfix.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4135
Summary:
When an unconfirmed user tries to sign in or reset their password, the email
verification flow is now automatically restarted. Prior to this change, an
unconfirmed user error was shown in the client.
Test Plan: Deployment tests.
Reviewers: paulfitz, dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4133
Summary: The visitorId is added to telemetry events from Matomo's cookie, if present.
Test Plan: Manual.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D4132
Summary:
Selection in GridView wasn't updated when fields were removed, and the selected
column index was out of bounds.
Test Plan: New test added
Reviewers: JakubSerafin
Reviewed By: JakubSerafin
Differential Revision: https://phab.getgrist.com/D4137
This change makes builtin custom widget bundles work on grist-electron,
by finding the package in a slightly more flexible way.
It also includes a related change to make a widget manifest fetched
from the network optional if a flag is present, with an error being
logged rather than thrown. This could make it harder to track down
why custom widgets aren't available, but makes it easier to make
grist-electron work (including calendars) when the network is shut off.
Ideally we'd do something fancier when we can.
Summary:
- "Add Column" -> "Add column"
- "Detect Duplicates in" -> "Detect duplicates in"
- "Last Updated At" -> "Last updated at"
- "Created By" (At) -> "Created by" (at)
- "Last Updated By" -> "Last updated by"
Test Plan: I looked at menu and cannot see any more capital letters anywhere other that begining of the menu position.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4127
Summary:
When the 'new' row of a table is selected, another table filter linked to the first shows no data. This diff ensures that a third table filtered by the second also shows no data, i.e. that it behaves the same as if the second table was also on the 'new' row. Video of the bug: https://grist.slack.com/archives/C069RUP71/p1692622810900179
The functional code is copied almost verbatim from https://github.com/gristlabs/grist-core/pull/666 by @jvorob which was working correctly. A comment there mentioned a possible bug where:
> ...you can have the grayed-out "No row selected" text from disableEditing but still have rows showing up in the section. Haven't been able to reproduce...
I noticed this behaviour when I copied only part of the fix, but it disappeared after copying the whole thing, so it seems likely to me that this is why it couldn't be reproduced.
Test Plan: Added a new nbrowser test with a new fixture, which also tests filter link chains and selecting the new row more generally, since I couldn't find other tests of this.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: jvorob
Differential Revision: https://phab.getgrist.com/D4126
Summary: Descriptions can now be set on Raw Data table sections.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4131
Summary:
Fixes bug described in https://grist.slack.com/archives/C069RUP71/p1699643458649019
Decodes cell values obtained from `InfoView.get` when evaluating user-defined ACL formulas, i.e. the result of `rec.foo` in such a formula. In particular this is so that `rec.some_list` loses the leading `L` type code and behaves sensibly in an expression like `thing in rec.some_list`.
`InfoView.get` is called in many places, but for every usage I found other than here, leaving the cell values encoded was best.
Test Plan: Added two unit server tests. The first is for the main bug involving lists. The second checks the only other plausible way I could think of that this change affects behaviour, and it seems to be for the better since both tests failed before. Most operations involving non-primitive cell values don't do anything sensible with or without decoding, so behaviour shouldn't change meaningfully in those cases.
Reviewers: georgegevoian, paulfitz
Reviewed By: georgegevoian, paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4123
Summary:
* Some tests needed updating because fake document ids in tests were changed to be valid urlIds (the existing ones were too short).
* urlId capture is tweaked to not allow hyphens, so some long login-related paths don't get confused with documents.
Test Plan: tests should pass again
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4134
Summary:
- Node has a strong recommendation to assume bad state and exit promptly on
unhandled exceptions and rejections. We follow it, and only make an effort to
clean up before exiting, and to log the error in a more standard way.
- The only case seen in recent month of an unhandled rejection was for
attempting to write overly large JSON to a Client websocket. Ensure that's
handled, and add a test case that artificially reproduces this scenario.
Test Plan:
Added a test case for failing write to Client, and a test case that unhandled
errors indeed kill the server but with an attempt at cleanup.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4124
Summary:
- Also, avoid scanning the database if relevant telemetry is off.
- Also, report time during breaks of reporting telemetry.
Test Plan: Tested manually in dev only that breaks are taken.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4125
The Calendar feature was implemented as a custom widget. To make it
available offline, we prepare a package that includes it, and add that
to Grist. The PluginManager is configured to find it.
An optional `GRIST_SKIP_BUNDLED_WIDGETS` flag is added to disable
widgets bundled this way from being used. This may be needed by
the tests in grist-widget to avoid getting an echo :-)
Summary: New Column menu was enhanced by "add column with type" and "add formula column" options. First one allow user to chose the type of newly created column, to save time for selecting this option in creator menu. "Add formula column" opens formula editor in popup state right after creating the column. In this case, renaming column popup is ignored to not overburden user with to many popup at once.
Test Plan: new nbrowser test was added to check validity of menu items, and output of menu action - if columns have given types or if formula editor popup is opened and functionin, accordingly.
Reviewers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4113
Summary:
Adds remaining functionality, fixes, and polish to Record Cards and
removes their feature flag, enabling them by default.
Test Plan: Tests deferred; will be included in a follow-up diff.
Reviewers: jarek, paulfitz
Reviewed By: jarek
Subscribers: paulfitz, jarek
Differential Revision: https://phab.getgrist.com/D4121
* support GRIST_OIDC_SP_PROFILE_NAME_ATTR, defaulting to the concatenation of "given_name" + "family_name" or the "name" attribute.
* support GRIST_OIDC_SP_PROFILE_EMAIL_ATTR, defaulting to "email".
* support GRIST_OIDC_IDP_SKIP_END_SESSION_ENDPOINT: If set to "true", will not attempt to call the IdP's end_session_endpoint. Fail early if the endpoint does not exist, and this variable isn't set.
The last part is because some IdPs like Gitlab do not provide end_session_endpoint. In such cases, GRIST_OIDC_IDP_SKIP_END_SESSION_ENDPOINT=true should be set to have the Grist logout button only log out of Grist, and not out of the IdP.
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary:
Adds a new Record Card view section to each non-summary table, which can be from opened from various parts of the Grist UI to view and edit records in a popup card view.
Work is still ongoing, so the feature is locked away behind a flag; follow-up work is planned to finish up the implementation and add end-to-end tests.
Test Plan: Python and server tests. Browser tests will be included in a follow-up.
Reviewers: jarek, paulfitz
Reviewed By: jarek
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4114
Summary:
Also update error handling in NSandbox initialization to avoid node
exiting when sandbox can't be created.
Info on oom_score and choom: https://man7.org/linux/man-pages/man1/choom.1.html
Empirically, in docker and under fargate, oom_score (visible in
/proc/PID/oom_score) seems to be calculated approximately as
((ProcessRSS / TotalMemory * 1000 + 999 + oom_score_adj) * 2/3)
though this doesn't correspond to any documentation I could find.
In addition, under docker / fargate it does not work to set oom_score_adj (with choom or via /proc/PID/oom_score_adj) to a negative value to give priority to a node process. That's why this diff adjust the score up for sandbox processes instead.
Test Plan:
Checked that grist-omnibus built with this change respects the
variable, and sandbox processes end up with higher oom_score values.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4112
Summary: Adds new telemetry events and a flag for whether an event originated from a team site.
Test Plan: Manual.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: jarek, dsagal
Differential Revision: https://phab.getgrist.com/D4105
* Fix OIDC redirects from team site to personal page after login #740
Also:
- compare state in session and state passed through parameters
(otherwise the state won't have any effect regarding the security).
- delete the session even after an authentication failure
* More logs for OIDC #740
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary: Some untagged assets on the plugin port could be a problem if that port is merged with the regular Grist app port, so we nest them within a non-conflicting path (/plugins/assets).
Test Plan: see if a test fails anywhere
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4116
Summary: Call a new user action `RemoveTransformColumns` in ActiveDoc shutdown.
Test Plan: Added nbrowser test
Reviewers: georgegevoian, paulfitz
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4107
Summary:
This removes problematic code that was holding a HomeDB transaction while applying user actions which could hang indefinitely, especially if the webhook queue is full as in https://grist.slack.com/archives/C05DBJ6LA1F/p1698159750945949.
The discussion about adding this code is here: https://phab.getgrist.com/D3821#inline-45054
The initial motivation was to roll back HomeDB changes if something went wrong while applying user actions, to avoid saving only part of the changes the user requested. I think it's actually fine to just allow such a partial save to happen - I don't see anything particularly undesirable about keeping an update to the webhook URL if other updates requested by the user didn't also get applied, as the fields don't affect each other.
The comment approving the transaction approach said "so we shouldn't end up leave the transaction hanging around too long" which has been falsified.
It looks like there was also some desire to prevent a mess caused by multiple simultaneous calls to this endpoint, which the transaction may have helped with a little, but didn't really seem like a solution. Comments in `Triggers.ts` also mention fears of race conditions when clearing (some of) the queue and the need for some locking. So I wrapped all webhook-related endpoints in a simple `Mutex` held by the `ActiveDoc` to prevent simultaneous changes. I *think* this is a good thing. These endpoints shouldn't be called frequently enough to create a performance issue, and this shouldn't affect actually sending webhook events when records are added/updated. And it does seem like interleaving calls to these endpoints could cause very weird problems.
Test Plan: Nothing yet, I'd like to hear if others think this is sensible.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4111
Summary:
When action is pruned by ACL rules, the rule engine sometimes
rewrites actions, setting NULLs for boolean columns as a default value.
Null was not considered a valid value and it resulted in a bug in the UI
where the editor for the Toggle column was not rendered.
Test Plan: Added new
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4110
Summary: Enabling the `GRIST_NEW_COLUMN_MENU` flag by default and removing it.
Test Plan: Existing
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4098
Summary:
Fix for a bug. Custom widget when collapsed and expanded was disconnecting from
Grist, as WidgetFrame was disposed to early.
Test Plan: Added new
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4109
Summary:
By default, only respect GRIST_FORWARD_AUTH_HEADER on login endpoints; sessions are used elsewhere.
With GRIST_IGNORE_SESSION, do not use sessions, and respect GRIST_FORWARD_AUTH_HEADER on all endpoints.
GRIST_PROXY_AUTH_HEADER is now a synonym to GRIST_FORWARD_AUTH_HEADER.
Test Plan: Fixed tests. Tested first approach (no GRIST_IGNORE_SESSION) with grist-omnibus manually. Tested the second approach (with GRIST_IGNORE_SESSION) with a Apache-based setup enforcing http basic auth on all endpoints.
Reviewers: paulfitz, georgegevoian
Reviewed By: paulfitz, georgegevoian
Differential Revision: https://phab.getgrist.com/D4104
Summary: Also fixes a few bugs found along the way, particularly that webhook payloads could contain stale data.
Test Plan: Added an nbrowser test, made existing test a bit more detailed.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4102
Summary:
Headings 4, 5, and 6 are now properly visible in dark mode. Additionally,
pre-formatted text and code blocks have improved styling in dark mode.
Test Plan: Manual.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4108
The getHostType() now returns "native" when the host corresponds to the value of APP_DOC_INTERNAL_URL. T
While trying to scale, with a different internal and public URL for doc workers, and having configured the org to be specified in the path (GRIST_ORG_IN_PATH=true), the APP_DOC_INTERNAL_URL parameter was not treated as internal which made the connection between home server and doc workers impossible.
---------
https://github.com/gristlabs/grist-core/pull/715
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary: Adds a handful of new telemetry events, and makes a few tweaks to allow for better organization of telemetry.
Test Plan: Manual.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4100
Summary:
This enables tooltips in other Grist deployment types (e.g. grist-core). Previously,
most of these tooltips were only enabled in the SaaS offering of Grist.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D4097
Summary:
Recently, the server became more strict about not responding to
requests before being fully configured. This is a problem when
a doc worker is trying to check whether it has become available
from a load balancer.
This change gives access to health checks prior to configuration
being complete. Otherwise, app endpoints accessed before full
configuration return a 503.
A flag is added to /status to allow checking explicitly for
the server being ready and configured.
Test Plan: manual
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4103
This makes a few refinements to bundling widgets:
* A widget with `published: false` is not shown in the
custom widget dropdown in the UI. This is so widgets
can be bundled with the app for "native" use (like the
calendar widget) without immediately resulting in an
extra listing in the UI. (There are improvements we'd
like to make to the UI to better communicate widget
provenance and quality eventually, which would be a
helpful alternative to just a binary flag.)
* A relative path to the custom widget manifest is
respected. This will make the bundling process marginally
neater.
Summary:
When converting changing the type of Any column, try to guess
the widgetOptions. Especially important for choice and choiceList types.
Test Plan: Existing
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D4088
Summary:
This adds support for bundling custom widgets with the Grist app, as follows:
* Adds a new `widgets` component to plugins mechanism.
* When a set of widgets is provided in a plugin, the html/js/css assets for those widgets are served on the existing untrusted user content port.
* Any bundled `grist-plugin-api.js` will be served with the Grist app's own version of that file. It is important that bundled widgets not refer to https://docs.getgrist.com for the plugin js, since they must be capable of working offline.
* The logic for configuring that port is updated a bit.
* I removed the CustomAttachedView class in favor of applying settings of bundled custom widgets more directly, without modification on view.
Any Grist installation via docker will need an extra step now, since there is an extra port that needs exposing for full functionality. I did add a `GRIST_TRUST_PLUGINS` option for anyone who really doesn't want to do this, and would prefer to trust the plugins and have them served on the same port.
Actually making use of bundling will be another step. It'll be important to mesh it with our SaaS's use of APP_STATIC_URL for serving most static assets.
Design sketch: https://grist.quip.com/bJlWACWzr2R9/Bundled-custom-widgets
Test Plan: added a test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4069
