Commit Graph

498 Commits

Author SHA1 Message Date
Dmitry S
be0b4a1968 (core) Use slugify module to construct slugs in urlIds, addressing a long-standing TODO.
Summary:
This offers better handling for accented and non-English characters,
e.g. "Événements" becomes "Evenements" (rather than "vnements") and
"таблиця" becomes "tablicya" rather than an empty string.

Test Plan:
Added a test case. Existing documents will auto-redirect to
newly-generated names. In cases where it's different, that's probably for the
best.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4176
2024-01-29 10:02:27 -05:00
Paul Fitzpatrick
fb276bade7 (core) updates from grist-core 2024-01-29 09:16:55 -05:00
Jarosław Sadziński
372d86618f (core) Required fields in forms and bug fixes
Summary:
- Adding little green asterisk at the end of field title.
- Fixing bug on columns component. Adding paragraph as a column and then selecting it was throwing error in the RightPanel
- Fixing boolean column bug in the editor
- Adding (--Choose--) placeholder for dropdowns
- Fixing columns logic: Dragging and dropping columns can unexpectedly add more columns.
- Added favicon and default page title
- Added svg to sync file for electron.

Test Plan: Updated

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D4172
2024-01-24 18:16:48 +01:00
George Gevoian
6cb8614017 (core) Polish forms
Summary:
  - Updates styling of form submitted page.
  - Tweaks styling of checkboxes, labels, and questions on form page.
  - Adds new form 404 page.
  - Adds checkbox to not show warning again when publishing or un-publishing a form.
  - Excludes formula, hidden, and attachment columns in submitted form data.
  - Adds placeholder text to form configuration inputs.
  - Improves dark mode styling in Form widget.
  - Updates default title and description of new forms.
  - Updates styling of Form widget buttons.
  - Fixes form success text input handling.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4170
2024-01-24 02:18:03 -08:00
George Gevoian
b77c762358 (core) Add sign-up and sharing/invite telemetry
Summary:
Enhances sign-up telemetry with login and verification method metadata, and
adds UTM parameters to SendGrid invite email links and Grist document links.

Test Plan: Server and manual.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D4169
2024-01-23 22:30:16 -08:00
Paul Fitzpatrick
dba3a59486 (core) fix form URL when team is encoded in domain
Summary:
This moves the `formUrl` logic to `encodeUrl`, which is more
aware of how the URL is constructed than UserAPI. UserAPI can
only reliably construct API URLs.

Test Plan: extended tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: georgegevoian

Differential Revision: https://phab.getgrist.com/D4171
2024-01-23 16:54:02 -05:00
Jarosław Sadziński
95c0441d84 (core) Form kanban tasks
Summary:
- Open all links in a new tab
- Excluding not filled columns (to fix trigger formulas)
- Fixed Ref/RefList submission
- Removing redundant type definitions for Box
- Adding header menu item
- Default empty values in select control

Test Plan: Updated

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4166
2024-01-23 22:11:48 +01:00
Florent
5533b9b7ee
Fix server crash when client passes malformed JSON (#826)
* Fix server crash when client passes malformed JSON

* Take remarks into account

---------

Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2024-01-23 12:07:39 -05:00
Jarosław Sadziński
0aad09a4ed (core) Forms improvements
Summary:
Forms improvements and following new design
- New headers
- New UI
- New right panel options

Test Plan: Tests updated

Reviewers: georgegevoian, dsagal

Reviewed By: georgegevoian

Subscribers: dsagal, paulfitz

Differential Revision: https://phab.getgrist.com/D4158
2024-01-19 10:34:03 +01:00
George Gevoian
b82209b458 (core) Fix filtering regression
Summary:
Fixes a recent regression that would cause a record to be erroneously filtered
out whenever it was updated from a linked view.

Test Plan: Browser test.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4163
2024-01-18 13:32:36 -08:00
George Gevoian
e12471347b (core) Form Publishing
Summary:
Adds initial implementation of form publishing, built upon WYSIWYS shares.

A simple UI for publishing and unpublishing forms is included.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: paulfitz, jarek

Differential Revision: https://phab.getgrist.com/D4154
2024-01-12 11:58:12 -08:00
Paul Fitzpatrick
eee29ee0a0 (core) add Shares migration to rollback test
Summary:
adds a newly-added migration to a list in a test for explicit
rollback testing

Test Plan: updates a test

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: georgegevoian

Differential Revision: https://phab.getgrist.com/D4161
2024-01-11 11:48:38 -05:00
Dmitry S
527e9670ef (core) Include linking rowIds into remembered cursor position and anchor links.
Summary:
When linking using a Reference List column, there may be multiple source
records that show the same target record. With this change, we remember those
(rather than just pick one that shows the target record).

Test Plan: Added a browser test.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4140
2024-01-09 09:50:31 -05:00
Paul Fitzpatrick
a311b8b3e5 (core) updates from grist-core 2024-01-08 09:19:50 -05:00
Paul Fitzpatrick
15a73fdaf0 (core) repair a migration test when running against postgres
Summary:
This is fixing some fall-out from some environmental changes after
another batch of tests were moved to grist-core.

Test Plan: this is fixing a test, no app changes

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4156
2024-01-04 13:35:55 -05:00
Paul Fitzpatrick
2a206dfcf8 (core) add initial support for special shares
Summary:
This gives a mechanism for controlling access control within a document that is distinct from (though implemented with the same machinery as) granular access rules.

It was hard to find a good way to insert this that didn't dissolve in a soup of complications, so here's what I went with:
 * When reading rules, if there are shares, extra rules are added.
 * If there are shares, all rules are made conditional on a "ShareRef" user property.
 * "ShareRef" is null when a doc is accessed in normal way, and the row id of a share when accessed via a share.

There's no UI for controlling shares (George is working on it for forms), but you can do it by editing a `_grist_Shares` table in a document. Suppose you make a fresh document with a single page/table/widget, then to create an empty share you can do:

```
gristDocPageModel.gristDoc.get().docData.sendAction(['AddRecord', '_grist_Shares', null, {linkId: 'xyz', options: '{"publish": true}'}])
```

If you look at the home db now there should be something in the `shares` table:

```
$ sqlite3 -table landing.db "select * from shares"
+----+------------------------+------------------------+--------------+---------+
| id |          key           |         doc_id         |   link_id    | options |
+----+------------------------+------------------------+--------------+---------+
| 1  | gSL4g38PsyautLHnjmXh2K | 4qYuace1xP2CTcPunFdtan | xyz | ...      |
+----+------------------------+------------------------+--------------+---------+
```

If you take the key from that (gSL4g38PsyautLHnjmXh2K in this case) and replace the document's urlId in its URL with `s.<key>` (in this case `s.gSL4g38PsyautLHnjmXh2K` then you can use the regular document landing page (it will be quite blank initially) or API endpoint via the share.

E.g. for me `http://localhost:8080/o/docs/s0gSL4g38PsyautLHnjmXh2K/share-inter-3` accesses the doc.

To actually share some material - useful commands:

```
gristDocPageModel.gristDoc.get().docData.getMetaTable('_grist_Views_section').getRecords()
gristDocPageModel.gristDoc.get().docData.sendAction(['UpdateRecord', '_grist_Views_section', 1, {shareOptions: '{"publish": true, "form": true}'}])
gristDocPageModel.gristDoc.get().docData.getMetaTable('_grist_Pages').getRecords()
gristDocPageModel.gristDoc.get().docData.sendAction(['UpdateRecord', '_grist_Pages', 1, {shareRef: 1}])
```

For a share to be effective, at least one page needs to have its shareRef set to the rowId of the share, and at least one widget on one of those pages needs to have its shareOptions set to {"publish": "true", "form": "true"} (meaning turn on sharing, and include form sharing), and the share itself needs {"publish": true} on its options.

I think special shares are kind of incompatible with public sharing, since by their nature (allowing access to all endpoints) they easily expose the docId, and changing that would be hard.

Test Plan: tests added

Reviewers: dsagal, georgegevoian

Reviewed By: dsagal, georgegevoian

Subscribers: jarek, dsagal

Differential Revision: https://phab.getgrist.com/D4144
2024-01-04 05:57:38 -05:00
Florent
837597cd55
Fix deadlock with webhooks on document load #799 (#812) 2024-01-03 21:47:53 +02:00
Florent
6722512d96
Completely ignored disabled webhooks (#800) 2024-01-03 20:06:38 +02:00
Paul Fitzpatrick
f079d4b340 (core) updates from grist-core 2024-01-02 08:00:39 -05:00
Alex Hall
225a76c9cb (core) Don't throw error in onRecord(s) for insufficient access for includeColumns
Summary:
This removes checking for full access in `onRecord/onRecords` when `includeColumns` is a non-default value. The check had two problems:

1. It relied on the access level being present in the URL query parameters, which doesn't work if the page has redirected. See the discussion in https://grist.slack.com/archives/C0234CPPXPA/p1702576602615509. There seems to be no way to reliably and synchronously check the access level.
2. Calling `onRecords` before `ready` and forgetting to handle an error from the access check meant that `ready` wouldn't be called, so Grist couldn't request the correct access level from the user. I made this mistake and it seems like a nasty footgun.

Ultimately this has no effect on security, as an error will still be raised, but in a place where the widget developer can't catch it. They'll still see an error message in the console, and they can still check the access level reliably using `onOptions`, so I think this is OK.

Test Plan: Updated nbrowser test

Reviewers: georgegevoian, paulfitz

Reviewed By: georgegevoian, paulfitz

Differential Revision: https://phab.getgrist.com/D4145
2023-12-30 10:16:40 +02:00
Paul Fitzpatrick
e0d44eff1f
set TEST_CLEAN_DATABASE when running server tests (#806)
After adding a batch of new server tests, some interactions between
tests have shown up via a shared database. This sets an existing flag
for dealing with this problem, that is used during browser tests but
hadn't been needed before for server tests.
2023-12-27 09:56:59 -05:00
Paul Fitzpatrick
145138b7e9 (core) move apiserver tests to core, disentangling notifier+billing parts
Summary:
This moves some more tests to core that would be useful for ANCT,
which had been stuck in grist-saas due to some entanglements with
sendgrid and billing. For sendgrid, I've moved around just enough
material to permit the tests to run mostly unchanged. Ideally
the interface to a notification system would be generalized, but
that's a bigger project.

Test Plan:
checked that tests are likely to run as expected
in core using preview laid out by ./buildtools/build_core.sh

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4149
2023-12-26 08:43:12 -05:00
Jarosław Sadziński
a424450cbe (core) Forms feature
Summary:
A new widget type Forms. For now hidden behind GRIST_EXPERIMENTAL_PLUGINS().
This diff contains all the core moving parts as a serves as a base to extend this functionality
further.

Test Plan: New test added

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4130
2023-12-20 13:23:12 +01:00
Jarosław Sadziński
337757d0ba (core) Fix for linking issue.
Summary:
If linking state changes multiple times frequently the code that simulates async operation is
wrongly debounced, which causes inverted order of execution. This fix makes sure that only the last
call to filter function is used.

Test Plan: Adding new test

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D4139
2023-12-20 12:57:41 +01:00
Alex Hall
bd52665f96 (core) Allow adding rows to widgets filtered by a link using a formula column
Summary:
When a widget `A` is selected by a widget `B` so that `A` is filtered, adding a new row to `A` uses the values in the selected row of `B` and the columns relevant to the linking as default values for the new row. This ensures that the new row matches the current linking filter and remains visible. However this would previously cause a sandbox error when one of the linking columns was a formula column, which doesn't allow setting values. This diff ignores formula columns when picking default values.

Since the value of the formula column in the new row typically won't match the linking filter, extra measures are needed to avoid the new row immediately disappearing. Regular filters already have a mechanism for this, but I didn't manage to extend it to also work for linking. Thanks @dsagal for creating `UnionRowSource` (originally in D4017) which is now used as the solution for temporarily exempting rows from both kinds of filtering.

While testing, I also came across another bug in linking summary tables that caused incorrect filtering, which I fixed with some changes to `DynamicQuerySet`.

Test Plan: Extended an nbrowser test, which both tests for the main change as well as the secondary bugfix.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D4135
2023-12-18 20:28:41 +02:00
Florent
c7f060be7a
Force test:docker to run with English language (#788)
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-12-15 07:43:54 -05:00
George Gevoian
7e05284cf2 (core) Add shortcut for opening Record Card
Summary: Also adds tests for previously untested Record Card behavior.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4136
2023-12-08 11:32:45 -05:00
Jarosław Sadziński
d0318af39b (core) Fixing bug with hiding multiple columns
Summary:
Selection in GridView wasn't updated when fields were removed, and the selected
column index was out of bounds.

Test Plan: New test added

Reviewers: JakubSerafin

Reviewed By: JakubSerafin

Differential Revision: https://phab.getgrist.com/D4137
2023-12-08 10:20:43 +01:00
Jakub Serafin
8038ee881c (core) Capitalization in new colum menu
Summary:
- "Add Column" -> "Add column"
- "Detect Duplicates in" -> "Detect duplicates in"
- "Last Updated At" -> "Last updated at"
- "Created By" (At) -> "Created by" (at)
- "Last Updated By" -> "Last updated by"

Test Plan: I looked at menu and cannot see any more capital letters anywhere other that begining of the menu position.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D4127
2023-12-05 17:01:22 +01:00
Alex Hall
a1c62f32f4 (core) Fix selecting new row in chain of filter links
Summary:
When the 'new' row of a table is selected, another table filter linked to the first shows no data. This diff ensures that a third table filtered by the second also shows no data, i.e. that it behaves the same as if the second table was also on the 'new' row. Video of the bug: https://grist.slack.com/archives/C069RUP71/p1692622810900179

The functional code is copied almost verbatim from https://github.com/gristlabs/grist-core/pull/666 by @jvorob which was working correctly. A comment there mentioned a possible bug where:

> ...you can have the grayed-out "No row selected" text from disableEditing but still have rows showing up in the section. Haven't been able to reproduce...

I noticed this behaviour when I copied only part of the fix, but it disappeared after copying the whole thing, so it seems likely to me that this is why it couldn't be reproduced.

Test Plan: Added a new nbrowser test with a new fixture, which also tests filter link chains and selecting the new row more generally, since I couldn't find other tests of this.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: jvorob

Differential Revision: https://phab.getgrist.com/D4126
2023-12-05 16:56:13 +02:00
Dmitry S
09c84734db (core) Enable the 'none' option in ColorSelect for cell and header text styles
Test Plan: Added a test case

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4120
2023-12-04 19:13:52 -05:00
George Gevoian
cc56e91f5b (core) Allow descriptions for Raw Data tables
Summary: Descriptions can now be set on Raw Data table sections.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4131
2023-12-04 16:52:56 -05:00
Alex Hall
887717bb15 (core) Decode cell values to prevent working around rule using 'in' on lists
Summary:
Fixes bug described in https://grist.slack.com/archives/C069RUP71/p1699643458649019

Decodes cell values obtained from `InfoView.get` when evaluating user-defined ACL formulas, i.e. the result of `rec.foo` in such a formula. In particular this is so that `rec.some_list` loses the leading `L` type code and behaves sensibly in an expression like `thing in rec.some_list`.

`InfoView.get` is called in many places, but for every usage I found other than here, leaving the cell values encoded was best.

Test Plan: Added two unit server tests. The first is for the main bug involving lists. The second checks the only other plausible way I could think of that this change affects behaviour, and it seems to be for the better since both tests failed before. Most operations involving non-primitive cell values don't do anything sensible with or without decoding, so behaviour shouldn't change meaningfully in those cases.

Reviewers: georgegevoian, paulfitz

Reviewed By: georgegevoian, paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4123
2023-12-04 23:34:08 +02:00
Dmitry S
4d9bbf6263 (core) Exit more cleanly on unhandled errors, and handle errors writing to Clients.
Summary:
- Node has a strong recommendation to assume bad state and exit promptly on
  unhandled exceptions and rejections. We follow it, and only make an effort to
  clean up before exiting, and to log the error in a more standard way.

- The only case seen in recent month of an unhandled rejection was for
  attempting to write overly large JSON to a Client websocket. Ensure that's
  handled, and add a test case that artificially reproduces this scenario.

Test Plan:
Added a test case for failing write to Client, and a test case that unhandled
errors indeed kill the server but with an attempt at cleanup.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4124
2023-12-01 09:42:00 -05:00
Florent
cf0cbb404e
Allow URLs with only a docID #768 (#771)
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-11-29 15:13:29 -05:00
jarek
7d57114d21
Fixing widget selection (#777) 2023-11-27 13:45:38 +01:00
Jakub Serafin
c04f61738c (core) "add column with type" and "add formula column" options in new column menu
Summary: New Column menu was enhanced by "add column with type" and "add formula column" options. First one allow user to chose the type of newly created column, to save time for selecting this option in creator menu. "Add formula column" opens formula editor in popup state right after creating the column. In this case, renaming column popup is ignored to not overburden user with to many popup at once.

Test Plan: new nbrowser test was added to check validity of menu items, and output of menu action - if columns have given types or if formula editor popup is opened and functionin, accordingly.

Reviewers: georgegevoian

Differential Revision: https://phab.getgrist.com/D4113
2023-11-23 14:31:27 +01:00
George Gevoian
707a8c7b32 (core) Enable Record Cards
Summary:
Adds remaining functionality, fixes, and polish to Record Cards and
removes their feature flag, enabling them by default.

Test Plan: Tests deferred; will be included in a follow-up diff.

Reviewers: jarek, paulfitz

Reviewed By: jarek

Subscribers: paulfitz, jarek

Differential Revision: https://phab.getgrist.com/D4121
2023-11-21 16:49:41 -05:00
Paul Fitzpatrick
cea0404a22 (core) updates from grist-core 2023-11-20 11:28:50 -05:00
George Gevoian
caf830db08 (core) Record Cards
Summary:
Adds a new Record Card view section to each non-summary table, which can be from opened from various parts of the Grist UI to view and edit records in a popup card view.

Work is still ongoing, so the feature is locked away behind a flag; follow-up work is planned to finish up the implementation and add end-to-end tests.

Test Plan: Python and server tests. Browser tests will be included in a follow-up.

Reviewers: jarek, paulfitz

Reviewed By: jarek

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4114
2023-11-19 20:12:37 -05:00
Florent
7bc862fb02
Add header=colId option for the table-schema API #719 (#749) 2023-11-17 17:45:15 +02:00
Paul Fitzpatrick
12317097f7 (core) move some untagged assets served on plugin port
Summary: Some untagged assets on the plugin port could be a problem if that port is merged with the regular Grist app port, so we nest them within a non-conflicting path (/plugins/assets).

Test Plan: see if a test fails anywhere

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4116
2023-11-14 15:58:00 -05:00
Alex Hall
5197891427 (core) Remove transform columns on shutdown
Summary: Call a new user action `RemoveTransformColumns` in ActiveDoc shutdown.

Test Plan: Added nbrowser test

Reviewers: georgegevoian, paulfitz

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4107
2023-11-14 22:31:34 +02:00
Paul Fitzpatrick
68801474b1 (core) updates from grist-core 2023-11-13 07:58:49 -05:00
Jarosław Sadziński
3c219e05f6 (core) Removing the new menu flag
Summary: Enabling the `GRIST_NEW_COLUMN_MENU` flag by default and removing it.

Test Plan: Existing

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4098
2023-11-08 07:37:54 +01:00
Jarosław Sadziński
9262e1f1ef (core) Fixing bug with collapsed custom widget.
Summary:
Fix for a bug. Custom widget when collapsed and expanded was disconnecting from
Grist, as WidgetFrame was disposed to early.

Test Plan: Added new

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4109
2023-11-08 07:36:21 +01:00
Dmitry S
3210eee24f (core) Revamp ForwardAuthLogin and unify with GRIST_PROXY_AUTH_HEADER
Summary:
By default, only respect GRIST_FORWARD_AUTH_HEADER on login endpoints; sessions are used elsewhere.

With GRIST_IGNORE_SESSION, do not use sessions, and respect GRIST_FORWARD_AUTH_HEADER on all endpoints.

GRIST_PROXY_AUTH_HEADER is now a synonym to GRIST_FORWARD_AUTH_HEADER.

Test Plan: Fixed tests. Tested first approach (no GRIST_IGNORE_SESSION) with grist-omnibus manually. Tested the second approach (with GRIST_IGNORE_SESSION) with a Apache-based setup enforcing http basic auth on all endpoints.

Reviewers: paulfitz, georgegevoian

Reviewed By: paulfitz, georgegevoian

Differential Revision: https://phab.getgrist.com/D4104
2023-11-07 16:30:49 -05:00
Alex Hall
b7e9d2705e (core) When a webhook is disabled, clear its queue
Summary: Also fixes a few bugs found along the way, particularly that webhook payloads could contain stale data.

Test Plan: Added an nbrowser test, made existing test a bit more detailed.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4102
2023-11-07 15:48:35 +02:00
George Gevoian
2f485c5435
Fix failing Docker tests (#724) 2023-11-06 12:49:32 -05:00
Paul Fitzpatrick
8053c81d02 (core) updates from grist-core 2023-11-06 08:20:57 -05:00