Commit Graph

407 Commits

Author SHA1 Message Date
Dmitry S
e170eef1a4 (core) A few assorted one-line fixes
Summary:
- Use newer flag in .npmrc to avoid warnings
- Fix check in WidgetRepository, useful for development but was broken
- Fix macSandboxExec for Macs that require libRosettaRuntime
- Make sure row count in Raw Data listing is visible when it takes more space

Test Plan: Tested manually

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3759
2023-01-09 23:31:04 -05:00
Paul Fitzpatrick
e6692c2793 (core) add a checkbox for owner "boss mode"
Summary:
Implement a checkbox that grants owners full access to tables by default, when creating new table/column rules.
 * Checkbox appears above default rules.
 * When set, a rule giving owners full access will be inserted in any new rule set started for tables or columns.
 * The checkbox can be expanded to allow customization of the rules.

https://gristlabs.getgrist.com/doc/check-ins/p/3#a1.s7.r2251.c19

Test Plan: added tests

Reviewers: jarek

Reviewed By: jarek

Subscribers: anaisconce

Differential Revision: https://phab.getgrist.com/D3756
2023-01-09 13:20:23 -05:00
Jarosław Sadziński
7ff2ca954c Adding eslint to github actions 2023-01-03 17:23:31 +01:00
Jarosław Sadziński
fd02a00a0e Fixing all eslint's reported error 2023-01-03 17:22:58 +01:00
Cyprien P
cabac3d9d8 (core) Adds new view as banner
Summary:
Diff removes view-as pill in the document breadcrumbs and add new view-as banner.

Note: Banners are still missing mechanism to handle several banners. As of now both doc-usage and view-as banners could show up at the same time.

Test Plan: Refactored existing test.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3732
2023-01-03 12:33:34 +01:00
Paul Fitzpatrick
9451fb9597 (core) updates from grist-core 2022-12-27 10:03:59 -05:00
Paul Fitzpatrick
103ebbb045
add MinIO tests (#381)
Extends workflow to test snapshots with minio.
2022-12-22 12:58:39 -05:00
jarek
506f61838a
Fixing time bug in webhook tests (#383)
Webhook tests were reusing date in logs, which caused a random failure in tests that checked updatedTime.
2022-12-22 12:15:06 -05:00
Paul Fitzpatrick
472a9a186e (core) control the distribution of attachment metadata
Summary:
for users who don't automatically have deep rights
to the document, provide them with attachment metadata only
for rows they have access to. This is a little tricky to
do efficiently. We provide attachment metadata when an
individual table is fetched, rather than on initial document
load, so we don't block that load on a full document scan.
We provide attachment metadata to a client when we see that
we are shipping rows mentioning particular attachments,
without making any effort to keep track of the metadata they
already have.

Test Plan: updated tests

Reviewers: dsagal, jarek

Reviewed By: dsagal, jarek

Differential Revision: https://phab.getgrist.com/D3722
2022-12-22 09:10:30 -05:00
Paul Fitzpatrick
e564d31582 (core) give preliminary support in core for storing snapshots in S3-compatible stores via minio-js client
Summary:
This is a first pass at snapshot support using the MinIO client, suitable
for use against a MinIO server or other S3-compatible storage (including
the original AWS S3).

In Grist Labs monorepo tests, it is run against AWS S3. It can be manually
configured to run again a MinIO server, and these tests pass. There are no
core tests just yet.

Next step would be to move external storage tests to core, and configure
workflow to run tests against a transient MinIO server.

Test Plan: applied same tests as for Azure and S3 (via AWS client)

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3729
2022-12-21 11:41:31 -05:00
Jarosław Sadziński
2a86cde474 (core) Hiding helper columns used for column transformation
Summary:
When a column is transformed, it creates two helper columns whose values are always
broadcasted to all clients. Now when there are some ACL rules, we are going to prune
those columns from messages sent to other connected clients.

Test Plan: Added new tests

Reviewers: dsagal, paulfitz

Reviewed By: dsagal, paulfitz

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3728
2022-12-21 10:07:17 +01:00
George Gevoian
c558800de5 (core) Avoid flagging support user as collaborator
Summary:
When initially added in the User Manager, the support user
(e.g. support@getgrist.com) was misleadingly being annotated as
a free collaborator. This fixes the annotation to be "Grist support"
instead.

Test Plan: Browser test.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3730
2022-12-14 01:23:49 -05:00
Jarosław Sadziński
629fcccd5a (core) Adding /webhooks endpoint
Summary:
- New /webhooks event that lists all webhooks in a document (available for owners),
- Monitoring webhooks usage and saving it in memory or Redis,
- Loosening _usubscribe API endpoint, so that the information returned from the /webhook endpoint is enough to unsubscribe,
- Owners can remove webhook without the unsubscribe key.

The endpoint lists all webhooks that are registered in a document, not just webhooks from a single table.
There are two status fields. First for the webhook, second for the last request attempt.
Webhook can have 5 statuses: 'idle', 'sending', 'retrying', 'postponed', 'error', which roughly describes what the
sendLoop is currently doing. The 'error' status describes a situation when all request attempts failed and the queue needs
to be drained, so some requests were dropped.

The last request status can only be: 'success', 'failure' or 'rejected'. Rejected means that the last batch was dropped because the
queue was too long.

Test Plan: New and updated tests

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3727
2022-12-13 22:46:40 +01:00
George Gevoian
e146f95c1c (core) Add new UI for writing memos
Summary:
Adds a new UI for writing access rule memos.

Migrates old memos (written as Python comments) to the new UI.

Test Plan: Browser and migration tests.

Reviewers: jarek, dsagal

Reviewed By: jarek

Subscribers: dsagal, paulfitz

Differential Revision: https://phab.getgrist.com/D3726
2022-12-12 17:52:01 -05:00
Paul Fitzpatrick
ebaf04dace (core) add buttons to delete bad rules
Summary:
When access rules refer to tables and/or columns that no longer exist, offer convenient buttons to remove these rules.

It could alternatively be useful to generate errors when deleting tables or columns that are mentioned in access rules, and refuse to do so unless the access rules are updated first.

Test Plan: added and updated tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3718
2022-12-05 11:49:41 -05:00
Paul Fitzpatrick
8c610dcb33 (core) updates from grist-core 2022-12-05 09:24:06 -05:00
Paul Fitzpatrick
bd762628e4 (core) confirm owner's right to download snapshots
Summary:
All users are treated as viewers for snapshot documents, since they
cannot reasonably be edited. This is a bit dubious and confusing now
that granular access rules exist. More urgently, owners of the trunk
document may be locked out of downloading a snapshot, and so also
locked out of replacing the trunk with a snapshot. This diff
explicitly gives an owner of a trunk document the right to download
its snapshots.

Test Plan: updated a snapshots test to something that fails without this diff

Reviewers: dsagal, georgegevoian

Reviewed By: dsagal, georgegevoian

Subscribers: jarek, dsagal

Differential Revision: https://phab.getgrist.com/D3721
2022-12-05 09:01:45 -05:00
Jarosław Sadziński
92d4fca855 (core) Adding DELETE /api/docs/webhooks/queue endpoint to clear the queue
Summary:
Creating an API endpoint to cancel any queued webhook messages from
a document.

Test Plan: Updated

Reviewers: paulfitz, georgegevoian

Reviewed By: paulfitz, georgegevoian

Differential Revision: https://phab.getgrist.com/D3713
2022-12-01 12:23:19 +01:00
Jarosław Sadziński
601ba58a2e (core) Syncing db with data when actions are rejected
Summary:
Writing results of the undo action to a database when the undo was caused by rejecting due to ACL checks.
This ensures that DB and sanbox are in sync in case of non-deterministic formulas.

Test Plan: Updated

Reviewers: georgegevoian, dsagal

Reviewed By: georgegevoian, dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3695
2022-11-29 10:34:57 +01:00
Louis Delbosc
a4b1145605
Add GRIST_HELP_CENTER environment variable (#363) 2022-11-28 15:19:31 -05:00
Jarosław Sadziński
2ca407505b (core) Allow doc owners to view document usage regardless of access rules
Summary: Document usage is now available for owners regardless ACL rules

Test Plan: Updated

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3717
2022-11-23 17:56:53 +01:00
Paul Fitzpatrick
312d2331a8 (core) lock down modification of the _grist_Attachments table
Summary:
Rows in the _grist_Attachments table have a special lifecycle,
being created by a special method, and deleted via a special
process. All other modifications are now rejected, for simplicity.

Test Plan: added test

Reviewers: dsagal, jarek

Reviewed By: dsagal, jarek

Differential Revision: https://phab.getgrist.com/D3712
2022-11-22 11:30:54 -05:00
Paul Fitzpatrick
6061b67fd9 (core) make filtering of metadata consistent in presence of default access rules
Summary:
This brings the treatment of metadata updates being broadcast to a user
into line with how it is treated when they first open the document.
Specifically, this fixes a bug where, for a document with a default access
rule denying access to everything, a user would not receive any
metadata updates.

Test Plan: added test; existing tests pass

Reviewers: jarek, dsagal

Reviewed By: jarek, dsagal

Differential Revision: https://phab.getgrist.com/D3711
2022-11-17 16:01:16 -05:00
George Gevoian
1a6d427339 (core) Update sort and filter UI
Summary:
The sort and filter UI now has a more unified UI, with similar
capabilities that are accessible from different parts of Grist.
It's now also possible to pin individual filters to the filter bar,
which replaces the old toggle for showing all filters in the
filter bar.

Test Plan: Various tests (browser, migration, project).

Reviewers: jarek, dsagal

Reviewed By: jarek, dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3669
2022-11-17 15:33:45 -05:00
Jarosław Sadziński
af462fc938 (core) Fixing the ViewAs feature when the example user exists
Summary:
View as feature uses example.com emails for simulated users. This can break
when such a user already exists in the home db. Here we pretend that these users
don't exist during ACL checks.

Test Plan: Updated and existing

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3705
2022-11-17 16:50:45 +01:00
Paul Fitzpatrick
7b7b26c983 (core) limit access to list of snapshots for documents with granular access
Summary:
Snapshots can now only be listed for users with non-nuanced access
(no access rules, or owners on docs with rules). If a snapshot URL
leaks, or is shared by a user who can list snapshots, that URL
behaves as before -- it gives access to the snapshot according
to access rules in that snapshot.

Test Plan: added test

Reviewers: georgegevoian, dsagal

Reviewed By: georgegevoian, dsagal

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3698
2022-11-15 11:58:44 -05:00
Paul Fitzpatrick
ea71312d0e (core) deal with write access for attachments
Summary:
Attachments are a special case for granular access control. A user is now allowed to read a given attachment if they have read access to a cell containing its id. So when a user writes to a cell in an attachment column, it is important that they can only write the ids of cells to which they have access. This diff allows a user to add an attachment id in a cell if:

  * The user already has access to that a attachment via some existing cell, or
  * The user recently updated the attachment, or
  * The attachment change is from an undo/redo of a previous action attributed to that user

Test Plan: Updated tests

Reviewers: georgegevoian, dsagal

Reviewed By: georgegevoian, dsagal

Differential Revision: https://phab.getgrist.com/D3681
2022-11-15 09:52:32 -05:00
Paul Fitzpatrick
42c3568835 (core) be stricter when replacing documents in the presence of granular access rules
Summary:
The /replace endpoint was built with home-level access control in mind. Updates needed:
  * Only an owner can now replace a document. Only owners are permitted to change granular access rules, and a document replacement could change granular access rules.
  * For the document being substituted in: the user must have complete access to view all material within it.

Test Plan: extended test

Reviewers: georgegevoian, dsagal

Reviewed By: georgegevoian, dsagal

Differential Revision: https://phab.getgrist.com/D3694
2022-11-09 14:14:09 -05:00
Alex Hall
e590e65a3f (core) Allow requests from untrusted origins but without credentials
Summary:
Allow requests from untrusted origins instead of returning an error, but don't allow credentials (Cookie header) or API keys (Authorization header).

Allow setting the header `Content-type: application/json` as an alternative to `X-Requested-With: XMLHttpRequest` to make it easier for clients to make POST/PUT/PATCH/DELETE requests without authentication.

Discussion: https://grist.slack.com/archives/C0234CPPXPA/p1666355281535479

Test Plan: Added and updated DocApi tests. Tested manually how this affects requests made from a browser.

Reviewers: paulfitz, dsagal

Reviewed By: paulfitz, dsagal

Differential Revision: https://phab.getgrist.com/D3678
2022-11-03 13:33:23 +02:00
Jarosław Sadziński
24b1ca92d7 Fixing translation key for sendAppPage.ts 2022-10-27 10:50:51 +02:00
Louis Delbosc
eea2ef5cfb
Use url.hostname instead of url.host to allow host from environment variable (#326)
Co-authored-by <yohan.boniface@free.fr>
2022-10-25 14:59:17 -04:00
Paul Fitzpatrick
0c82b746d0 (core) updates from grist-core 2022-10-24 10:53:18 -04:00
Jarosław Sadziński
6460c22a89 (core) Changing shortcuts for adding and removing rows
Summary:
New shortcuts for removing and adding rows.
For adding a row we now have Mod+(Shift)+Enter
For removing rows we now have Mod+Delete/Mod+Backspace

Before removing rows, the user is prompted to confirm, this prompt
can be dismissed and this setting can be remembered. User needs
to confirm only when using shortcut.

Old shortcuts are still active and shows information about this change.
This information is shown only once, after this shortcuts have default
behavior (zooming).
New users don't see this explanation.

Test Plan: Updated

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3655
2022-10-21 18:45:25 +02:00
Alex Hall
62792329c3 (core) DocApi meta endpoints: GET /tables and POST/PATCH /tables and /columns
Summary:
Adds new API endpoints to list tables in a document and create or modify tables and columns. The request and response formats are designed to mirror the style of the existing `GET /columns` and `GET/POST/PATCH /records` endpoints.

Discussion: https://grist.slack.com/archives/C0234CPPXPA/p1665139807125649?thread_ts=1628957179.010500&cid=C0234CPPXPA

Test Plan: DocApi test

Reviewers: jarek

Reviewed By: jarek

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3667
2022-10-21 10:15:46 +02:00
jarek
4bb1d8c011
Merge pull request #312 from incubateur-territoires/arnaudpeich/Split_client_and_server_translations_organize_by_filename
Split client and server translations, organize by filename
2022-10-19 10:38:39 +02:00
Paul Fitzpatrick
bf24c29de4 (core) updates from grist-core 2022-10-17 10:53:21 -04:00
Jarosław Sadziński
bfd7243fe2 (core) Comments
Summary:
First iteration for comments system for Grist.
- Comments are stored in a generic metatable `_grist_Cells`
- Each comment is connected to a particular cell (hence the generic name of the table)
- Access level works naturally for records stored in this table
-- User can add/read comments for cells he can see
-- User can't update/remove comments that he doesn't own, but he can delete them by removing cells (rows/columns)
-- Anonymous users can't see comments at all.
- Each comment can have replies (but replies can't have more replies)

Comments are hidden by default, they can be enabled by COMMENTS=true env variable.
Some things for follow-up
- Avatars, currently the user's profile image is not shown or retrieved from the server
- Virtual rendering for comments list in creator panel. Currently, there is a limit of 200 comments.

Test Plan: New and existing tests

Reviewers: georgegevoian, paulfitz

Reviewed By: georgegevoian

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3509
2022-10-17 13:38:21 +02:00
Paul Fitzpatrick
ab3cdb62ac (core) make the support account configurable, and allow listing public sites
Summary:
This makes two small tweaks based on a user's questions about sharing
sites publicly for a self-managed installation:

 * The support user `support@getgrist.com` is made configurable with
   `GRIST_SUPPORT_EMAIL`. This came up because only the support user
   can share material with the special "everyone" user. This restriction
   was added to avoid spam.
 * Regardless of public sharing settings, for our SaaS we had
   decided not to list public sites to anonymous users. That is
   somewhat a question of taste, so a `GRIST_LIST_PUBLIC_SITES` flag
   is added to override this choice.

Public sharing isn't in a well polished state, and this diff doesn't
advance that, in fact it adds a new wrinkle :-/

Test Plan: existing tests pass; manual testing

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3663
2022-10-14 14:23:59 -04:00
Arnaud Peich
5f66a8f298
Return 403 error when origin is not trusted (#310) 2022-10-13 09:13:01 -04:00
Arnaud Peich
a44989e4dd Introduce translate helpers 2022-10-13 12:31:26 +02:00
Arnaud Peich
cc2a438fe5 Split client and server translations, organize by filename 2022-10-13 12:04:29 +02:00
Paul Fitzpatrick
6dd2068218 (core) show package.json version when hovering on Grist icon in grist-core
Summary:
This makes the version shown when hovering on the Grist icon equal
the version set in package.json, for a grist-core build. Previously
the number shown was a hard-coded placeholder.

The Grist SaaS build has some build machinery dealing with the
version number that should be unaffected by this change for now.

Test Plan: tested manually with build_core.sh

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3659
2022-10-12 16:02:01 -04:00
Jarosław Sadziński
9628253fd8 (core) Adding new column in users table "ref" with unique identifier.
Summary:
There is a new column in users table called ref (user reference).
It holds user's unique reference number that can be used for features
that require some kind of ownership logic (like comments).

Test Plan: Updated tests

Reviewers: georgegevoian, paulfitz

Reviewed By: georgegevoian, paulfitz

Differential Revision: https://phab.getgrist.com/D3641
2022-10-04 15:19:28 +02:00
Paul Fitzpatrick
433e1ecfc2 (core) updates from grist-core 2022-09-29 13:14:04 -04:00
Jarosław Sadziński
5219932a1f (core) i18
Summary:
Adding initial work for localization support.

Summary in https://grist.quip.com/OtZKA6RHdQ6T/Internationalization-and-Localization

Test Plan: Not yet

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3633
2022-09-29 18:02:09 +02:00
George Gevoian
cd64237dad (core) Allow duplicating tables from Raw Data page
Summary:
Adds a "Duplicate Table" menu option to the tables listed on
the Raw Data page. Clicking it opens a dialog that allows you to
make a copy of the table (with or without its data).

Test Plan: Python, server, and browser tests.

Reviewers: jarek, paulfitz

Reviewed By: jarek, paulfitz

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3619
2022-09-29 08:59:42 -07:00
Paul Fitzpatrick
0eb1fec3d7 (core) show any error when creating initial team site with GRIST_SINGLE_ORG
Summary: Also clarifies that only lowercase letters are accepted.

Test Plan: manual

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3647
2022-09-29 10:12:32 -04:00
Paul Fitzpatrick
20f935367b
remove old comment that gvisor is not available in grist-core (#295)
Removes a comment now that `gvisor` works fine with grist-core, and is packaged in the docker image. Reorders possible sandbox flavors to de-emphasize `pynbox` since it isn't packaged in the docker image.
2022-09-28 17:29:32 -04:00
Jarosław Sadziński
198beaab2a (core) Ref columns weren't filtered on csv/excel export for sections.
Summary:
Ref columns weren't filtred on section export.
Filters were applied to a display helper columns instead
of the actual columns.

Test Plan: Updated tests

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3644
2022-09-28 22:32:14 +02:00
Alex Hall
792565976a (core) Show example values in formula autocomplete
Summary:
This diff adds a preview of the value of certain autocomplete suggestions, especially of the form `$foo.bar` or `user.email`. The main initial motivation was to show the difference between `$Ref` and `$Ref.DisplayCol`, but the feature is more general.

The client now sends the row ID of the row being edited (along with the table and column IDs which were already sent) to the server to fetch autocomplete suggestions. The returned suggestions are now tuples `(suggestion, example_value)` where `example_value` is a string or null. The example value is simply obtained by evaluating (in a controlled way) the suggestion in the context of the given record and the current user. The string representation is similar to the standard `repr` but dates and datetimes are formatted, and the whole thing is truncated for efficiency.

The example values are shown in the autocomplete popup separated from the actual suggestion by a number of spaces calculated to:

1. Clearly separate the suggestion from the values
2. Left-align the example values in most cases
3. Avoid having so much space such that connecting suggestions and values becomes visually difficult.

The tokenization of the row is then tweaked to show the example in light grey to deemphasise it.

Main discussion where the above was decided: https://grist.slack.com/archives/CDHABLZJT/p1661795588100009

The diff also includes various other small improvements and fixes:

- The autocomplete popup is much wider to make room for long suggestions, particularly lookups, as pointed out in https://phab.getgrist.com/D3580#inline-41007. The wide popup is the reason a fancy solution was needed to position the example values. I didn't see a way to dynamically resize the popup based on suggestions, and it didn't seem like a good idea to try.
- The `grist` and `python` labels previously shown on the right are removed. They were not helpful (https://grist.slack.com/archives/CDHABLZJT/p1659697086155179) and would get in the way of the example values.
- Fixed a bug in our custom tokenization that caused function arguments to be weirdly truncated in the middle: https://grist.slack.com/archives/CDHABLZJT/p1661956353699169?thread_ts=1661953258.342739&cid=CDHABLZJT and https://grist.slack.com/archives/C069RUP71/p1659696778991339
- Hide suggestions involving helper columns like `$gristHelper_Display` or `Table.lookupRecords(gristHelper_Display=` (https://grist.slack.com/archives/CDHABLZJT/p1661953258342739). The former has been around for a while and seems to be a mistake. The fix is simply to use `is_visible_column` instead of `is_user_column`. Since the latter is not used anywhere else, and using it in the first place seems like a mistake more than anything else, I've also removed the function to prevent similar mistakes in the future.
- Don't suggest private columns as lookup arguments: https://grist.slack.com/archives/CDHABLZJT/p1662133416652499?thread_ts=1661795588.100009&cid=CDHABLZJT
- Only fetch fresh suggestions specifically after typing `lookupRecords(` or `lookupOne(` rather than just `(`, as this would needlessly hide function suggestions which could still be useful to see the arguments. However this only makes a difference when there are still multiple matching suggestions, otherwise Ace hides them anyway.

Test Plan: Extended and updated several Python and browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3611
2022-09-28 19:42:36 +02:00