Commit Graph

351 Commits

Author SHA1 Message Date
Cyprien P
a8827b152e (core) New document settings page
Summary:
This diff implement new page for document settings which replaces the old modal settings.
Diff also adds a new `Settings + API` page item below tools in the left panel that link to that very doc settings page.

Test Plan: Updated existing tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3761
2023-01-19 10:21:23 +01:00
Dmitry S
45c7602f49 (core) Implement checkbox for SchemaEdit permission in Access Rules UI.
Summary:
- Introduces a fictitious "*SPECIAL:SchemaEdit" resource in UI only.
- Hides "S" bit for the default rule section.
- Shows a checkbox UI similar to other checkboxes, with an additional
  dismissable warning.

Test Plan: Added a browser test

Reviewers: paulfitz, georgegevoian

Reviewed By: paulfitz, georgegevoian

Differential Revision: https://phab.getgrist.com/D3765
2023-01-18 10:55:44 -05:00
Cyprien P
18d016c745 (core) Fix date filter for DateTime columns.
Summary:
Date filter was not taking timezone correclty into account, which was
causing to wrong-inclusion and wrong-exclusion of dates near the
bounds.

Diff fixes that, it also bring little refactoring that hopefully clarifies things a little.

Test Plan: Includes brand new test for `app/common/ColumnFilterFunc`.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3763
2023-01-18 10:18:15 +01:00
George Gevoian
db64dfeef0 (core) Add tip for "Add New" button
Summary:
Adds a new tip for the doc menu's Add New button. The tip is
shown only when the current user is an editor or owner, and
the site is non-empty. The presence of welcome videos or
popups will also cause the tip to not be shown; it will instead
be shown the next time the doc menu is visited.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3757
2023-01-16 16:50:42 -08:00
Cyprien P
b7f65ff408 (core) Adds dots menu to access rules page item
Summary: In Access rules page item, adds “…” buttons that shows a menu of users to view-as:

Test Plan: Include new nbrowser test

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3751
2023-01-12 09:40:45 +01:00
George Gevoian
9e009bbab9 (core) Update memo notification styling
Summary: Updates CSS for memo notifications to match latest Figma designs.

Test Plan: N/A

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3737
2023-01-09 14:34:59 -05:00
Paul Fitzpatrick
e6692c2793 (core) add a checkbox for owner "boss mode"
Summary:
Implement a checkbox that grants owners full access to tables by default, when creating new table/column rules.
 * Checkbox appears above default rules.
 * When set, a rule giving owners full access will be inserted in any new rule set started for tables or columns.
 * The checkbox can be expanded to allow customization of the rules.

https://gristlabs.getgrist.com/doc/check-ins/p/3#a1.s7.r2251.c19

Test Plan: added tests

Reviewers: jarek

Reviewed By: jarek

Subscribers: anaisconce

Differential Revision: https://phab.getgrist.com/D3756
2023-01-09 13:20:23 -05:00
Jarosław Sadziński
b59829d57e (core) Updating dependencies for moment package
Summary:
moment.js version in yarn.lock file was outdated and contained
a set of typings that are diffrent from the one visible from grist-core.

Test Plan: Existing tests

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3760
2023-01-09 18:20:01 +01:00
Jarosław Sadziński
30600c5dc2 (core) Making makeT helper use the unprefixed key as a fallback
Summary:
By default the fallback contains the prefix, which doesn't
work well with a key based translations. Now makeT helper will fallback
to the passed key (without a prefix).

Test Plan: Added new client test

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3758
2023-01-05 16:25:19 +01:00
Jarosław Sadziński
fd02a00a0e Fixing all eslint's reported error 2023-01-03 17:22:58 +01:00
George Gevoian
77e85c132c
Fix control-border CSS regression (#388) 2023-01-03 10:16:23 -05:00
Cyprien P
cabac3d9d8 (core) Adds new view as banner
Summary:
Diff removes view-as pill in the document breadcrumbs and add new view-as banner.

Note: Banners are still missing mechanism to handle several banners. As of now both doc-usage and view-as banners could show up at the same time.

Test Plan: Refactored existing test.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3732
2023-01-03 12:33:34 +01:00
Paul Fitzpatrick
472a9a186e (core) control the distribution of attachment metadata
Summary:
for users who don't automatically have deep rights
to the document, provide them with attachment metadata only
for rows they have access to. This is a little tricky to
do efficiently. We provide attachment metadata when an
individual table is fetched, rather than on initial document
load, so we don't block that load on a full document scan.
We provide attachment metadata to a client when we see that
we are shipping rows mentioning particular attachments,
without making any effort to keep track of the metadata they
already have.

Test Plan: updated tests

Reviewers: dsagal, jarek

Reviewed By: dsagal, jarek

Differential Revision: https://phab.getgrist.com/D3722
2022-12-22 09:10:30 -05:00
Jarosław Sadziński
2a86cde474 (core) Hiding helper columns used for column transformation
Summary:
When a column is transformed, it creates two helper columns whose values are always
broadcasted to all clients. Now when there are some ACL rules, we are going to prune
those columns from messages sent to other connected clients.

Test Plan: Added new tests

Reviewers: dsagal, paulfitz

Reviewed By: dsagal, paulfitz

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3728
2022-12-21 10:07:17 +01:00
George Gevoian
e52e15591d (core) Add behavioral and coaching call popups
Summary:
Adds a new category of popups that are shown dynamically when
certain parts of the UI are first rendered, and a free coaching
call popup that's shown to users on their site home page.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3706
2022-12-20 09:49:41 -05:00
Cyprien P
620e86a9f1 (core) New date filter with a calendar view
Summary:
Implements the new date filtering panel. Design results from long
discussion between: Alex, Anais, Cyprien and Dmitry.

Test environment: https://grist-new-date-range-filter.fly.dev/

Test Plan: Include various new tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3720
2022-12-20 09:58:42 +01:00
George Gevoian
c558800de5 (core) Avoid flagging support user as collaborator
Summary:
When initially added in the User Manager, the support user
(e.g. support@getgrist.com) was misleadingly being annotated as
a free collaborator. This fixes the annotation to be "Grist support"
instead.

Test Plan: Browser test.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3730
2022-12-14 01:23:49 -05:00
George Gevoian
e146f95c1c (core) Add new UI for writing memos
Summary:
Adds a new UI for writing access rule memos.

Migrates old memos (written as Python comments) to the new UI.

Test Plan: Browser and migration tests.

Reviewers: jarek, dsagal

Reviewed By: jarek

Subscribers: dsagal, paulfitz

Differential Revision: https://phab.getgrist.com/D3726
2022-12-12 17:52:01 -05:00
Paul Fitzpatrick
ebaf04dace (core) add buttons to delete bad rules
Summary:
When access rules refer to tables and/or columns that no longer exist, offer convenient buttons to remove these rules.

It could alternatively be useful to generate errors when deleting tables or columns that are mentioned in access rules, and refuse to do so unless the access rules are updated first.

Test Plan: added and updated tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3718
2022-12-05 11:49:41 -05:00
Louis Delbosc
4116949ea5
Add translation for boolean value in toggle columns for text cell (#364) 2022-12-02 15:49:55 -05:00
Louis Delbosc
a4b1145605
Add GRIST_HELP_CENTER environment variable (#363) 2022-11-28 15:19:31 -05:00
Paul Fitzpatrick
312d2331a8 (core) lock down modification of the _grist_Attachments table
Summary:
Rows in the _grist_Attachments table have a special lifecycle,
being created by a special method, and deleted via a special
process. All other modifications are now rejected, for simplicity.

Test Plan: added test

Reviewers: dsagal, jarek

Reviewed By: dsagal, jarek

Differential Revision: https://phab.getgrist.com/D3712
2022-11-22 11:30:54 -05:00
George Gevoian
1a6d427339 (core) Update sort and filter UI
Summary:
The sort and filter UI now has a more unified UI, with similar
capabilities that are accessible from different parts of Grist.
It's now also possible to pin individual filters to the filter bar,
which replaces the old toggle for showing all filters in the
filter bar.

Test Plan: Various tests (browser, migration, project).

Reviewers: jarek, dsagal

Reviewed By: jarek, dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3669
2022-11-17 15:33:45 -05:00
Paul Fitzpatrick
ea71312d0e (core) deal with write access for attachments
Summary:
Attachments are a special case for granular access control. A user is now allowed to read a given attachment if they have read access to a cell containing its id. So when a user writes to a cell in an attachment column, it is important that they can only write the ids of cells to which they have access. This diff allows a user to add an attachment id in a cell if:

  * The user already has access to that a attachment via some existing cell, or
  * The user recently updated the attachment, or
  * The attachment change is from an undo/redo of a previous action attributed to that user

Test Plan: Updated tests

Reviewers: georgegevoian, dsagal

Reviewed By: georgegevoian, dsagal

Differential Revision: https://phab.getgrist.com/D3681
2022-11-15 09:52:32 -05:00
Jarosław Sadziński
7c9cb9843e (core) Revealing hidden pages with visible children.
Summary:
When a page is hidden, all its nested pages are shown as children of
a different page that happens to be before (as in pagePos) that page.

This diff shows those pages as CENSORED.

Test Plan: Updated

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3670
2022-10-31 14:02:38 +01:00
George Gevoian
3145af36c6 (core) Add dark mode for comments
Summary:
Also fixes the CSS for the page/workspace input so that it's always
readable in dark mode.

Test Plan: Tested manually.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3682
2022-10-25 08:19:06 -07:00
Jarosław Sadziński
6460c22a89 (core) Changing shortcuts for adding and removing rows
Summary:
New shortcuts for removing and adding rows.
For adding a row we now have Mod+(Shift)+Enter
For removing rows we now have Mod+Delete/Mod+Backspace

Before removing rows, the user is prompted to confirm, this prompt
can be dismissed and this setting can be remembered. User needs
to confirm only when using shortcut.

Old shortcuts are still active and shows information about this change.
This information is shown only once, after this shortcuts have default
behavior (zooming).
New users don't see this explanation.

Test Plan: Updated

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3655
2022-10-21 18:45:25 +02:00
George Gevoian
4c662253a9 (core) Add info and hover tooltips
Summary:
Adds tooltip buttons to various parts of the UI that either open a popup with
information when clicked, or show a label on hover.

Test Plan: Project tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3657
2022-10-20 15:17:44 -07:00
Jarosław Sadziński
bfd7243fe2 (core) Comments
Summary:
First iteration for comments system for Grist.
- Comments are stored in a generic metatable `_grist_Cells`
- Each comment is connected to a particular cell (hence the generic name of the table)
- Access level works naturally for records stored in this table
-- User can add/read comments for cells he can see
-- User can't update/remove comments that he doesn't own, but he can delete them by removing cells (rows/columns)
-- Anonymous users can't see comments at all.
- Each comment can have replies (but replies can't have more replies)

Comments are hidden by default, they can be enabled by COMMENTS=true env variable.
Some things for follow-up
- Avatars, currently the user's profile image is not shown or retrieved from the server
- Virtual rendering for comments list in creator panel. Currently, there is a limit of 200 comments.

Test Plan: New and existing tests

Reviewers: georgegevoian, paulfitz

Reviewed By: georgegevoian

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3509
2022-10-17 13:38:21 +02:00
Jarosław Sadziński
8be920dd25 (core) Multi-column configuration
Summary:
Creator panel allows now to edit multiple columns at once
for some options that are common for them. Options that
are not common are disabled.

List of options that can be edited for multiple columns:
- Column behavior (but limited to empty/formula columns)
- Alignment and wrapping
- Default style
- Number options (for numeric columns)
- Column types (but only for empty/formula columns)

If multiple columns of the same type are selected, most of
the options are available to change, except formula, trigger formula
and conditional styles.

Editing column label or column id is disabled by default for multiple
selection.

Not related: some tests were fixed due to the change in the column label
and id widget in grist-core (disabled attribute was replaced by readonly).

Test Plan: Updated and new tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3598
2022-10-17 09:51:19 +02:00
Paul Fitzpatrick
ab3cdb62ac (core) make the support account configurable, and allow listing public sites
Summary:
This makes two small tweaks based on a user's questions about sharing
sites publicly for a self-managed installation:

 * The support user `support@getgrist.com` is made configurable with
   `GRIST_SUPPORT_EMAIL`. This came up because only the support user
   can share material with the special "everyone" user. This restriction
   was added to avoid spam.
 * Regardless of public sharing settings, for our SaaS we had
   decided not to list public sites to anonymous users. That is
   somewhat a question of taste, so a `GRIST_LIST_PUBLIC_SITES` flag
   is added to override this choice.

Public sharing isn't in a well polished state, and this diff doesn't
advance that, in fact it adds a new wrinkle :-/

Test Plan: existing tests pass; manual testing

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3663
2022-10-14 14:23:59 -04:00
Paul Fitzpatrick
6dd2068218 (core) show package.json version when hovering on Grist icon in grist-core
Summary:
This makes the version shown when hovering on the Grist icon equal
the version set in package.json, for a grist-core build. Previously
the number shown was a hard-coded placeholder.

The Grist SaaS build has some build machinery dealing with the
version number that should be unaffected by this change for now.

Test Plan: tested manually with build_core.sh

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3659
2022-10-12 16:02:01 -04:00
George Gevoian
74c556ea85 (core) Fix CSS errors related to dark mode
Summary: Fixes a few cosmetic CSS bugs since dark mode landed.

Test Plan: Tested manually.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3649
2022-10-05 00:05:57 -07:00
Jarosław Sadziński
9628253fd8 (core) Adding new column in users table "ref" with unique identifier.
Summary:
There is a new column in users table called ref (user reference).
It holds user's unique reference number that can be used for features
that require some kind of ownership logic (like comments).

Test Plan: Updated tests

Reviewers: georgegevoian, paulfitz

Reviewed By: georgegevoian, paulfitz

Differential Revision: https://phab.getgrist.com/D3641
2022-10-04 15:19:28 +02:00
George Gevoian
364610c69d (core) Add cell selection summary
Summary:
Adds a cell selection summary to grid view that shows either a count or
sum of all the selected values. Implementation was done by Dmitry.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: paulfitz, dsagal, jarek

Differential Revision: https://phab.getgrist.com/D3630
2022-09-30 09:11:46 -07:00
Jarosław Sadziński
5219932a1f (core) i18
Summary:
Adding initial work for localization support.

Summary in https://grist.quip.com/OtZKA6RHdQ6T/Internationalization-and-Localization

Test Plan: Not yet

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3633
2022-09-29 18:02:09 +02:00
George Gevoian
cd64237dad (core) Allow duplicating tables from Raw Data page
Summary:
Adds a "Duplicate Table" menu option to the tables listed on
the Raw Data page. Clicking it opens a dialog that allows you to
make a copy of the table (with or without its data).

Test Plan: Python, server, and browser tests.

Reviewers: jarek, paulfitz

Reviewed By: jarek, paulfitz

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3619
2022-09-29 08:59:42 -07:00
Paul Fitzpatrick
0eb1fec3d7 (core) show any error when creating initial team site with GRIST_SINGLE_ORG
Summary: Also clarifies that only lowercase letters are accepted.

Test Plan: manual

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3647
2022-09-29 10:12:32 -04:00
Alex Hall
792565976a (core) Show example values in formula autocomplete
Summary:
This diff adds a preview of the value of certain autocomplete suggestions, especially of the form `$foo.bar` or `user.email`. The main initial motivation was to show the difference between `$Ref` and `$Ref.DisplayCol`, but the feature is more general.

The client now sends the row ID of the row being edited (along with the table and column IDs which were already sent) to the server to fetch autocomplete suggestions. The returned suggestions are now tuples `(suggestion, example_value)` where `example_value` is a string or null. The example value is simply obtained by evaluating (in a controlled way) the suggestion in the context of the given record and the current user. The string representation is similar to the standard `repr` but dates and datetimes are formatted, and the whole thing is truncated for efficiency.

The example values are shown in the autocomplete popup separated from the actual suggestion by a number of spaces calculated to:

1. Clearly separate the suggestion from the values
2. Left-align the example values in most cases
3. Avoid having so much space such that connecting suggestions and values becomes visually difficult.

The tokenization of the row is then tweaked to show the example in light grey to deemphasise it.

Main discussion where the above was decided: https://grist.slack.com/archives/CDHABLZJT/p1661795588100009

The diff also includes various other small improvements and fixes:

- The autocomplete popup is much wider to make room for long suggestions, particularly lookups, as pointed out in https://phab.getgrist.com/D3580#inline-41007. The wide popup is the reason a fancy solution was needed to position the example values. I didn't see a way to dynamically resize the popup based on suggestions, and it didn't seem like a good idea to try.
- The `grist` and `python` labels previously shown on the right are removed. They were not helpful (https://grist.slack.com/archives/CDHABLZJT/p1659697086155179) and would get in the way of the example values.
- Fixed a bug in our custom tokenization that caused function arguments to be weirdly truncated in the middle: https://grist.slack.com/archives/CDHABLZJT/p1661956353699169?thread_ts=1661953258.342739&cid=CDHABLZJT and https://grist.slack.com/archives/C069RUP71/p1659696778991339
- Hide suggestions involving helper columns like `$gristHelper_Display` or `Table.lookupRecords(gristHelper_Display=` (https://grist.slack.com/archives/CDHABLZJT/p1661953258342739). The former has been around for a while and seems to be a mistake. The fix is simply to use `is_visible_column` instead of `is_user_column`. Since the latter is not used anywhere else, and using it in the first place seems like a mistake more than anything else, I've also removed the function to prevent similar mistakes in the future.
- Don't suggest private columns as lookup arguments: https://grist.slack.com/archives/CDHABLZJT/p1662133416652499?thread_ts=1661795588.100009&cid=CDHABLZJT
- Only fetch fresh suggestions specifically after typing `lookupRecords(` or `lookupOne(` rather than just `(`, as this would needlessly hide function suggestions which could still be useful to see the arguments. However this only makes a difference when there are still multiple matching suggestions, otherwise Ace hides them anyway.

Test Plan: Extended and updated several Python and browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3611
2022-09-28 19:42:36 +02:00
Alex Hall
1864b7ba5d (core) Add BulkAddOrUpdateRecord action for efficiency
Summary:
This diff adds a new `BulkAddOrUpdateRecord` user action which is what is sounds like:

- A bulk version of the existing `AddOrUpdateRecord` action.
- Much more efficient for operating on many records than applying many individual actions.
- Column values are specified as maps from `colId` to arrays of values as usual.
- Produces bulk versions of `AddRecord` and `UpdateRecord` actions instead of many individual actions.

Examples of users wanting to use something like `AddOrUpdateRecord` with large numbers of records:

- https://grist.slack.com/archives/C0234CPPXPA/p1651789710290879
- https://grist.slack.com/archives/C0234CPPXPA/p1660743493480119
- https://grist.slack.com/archives/C0234CPPXPA/p1660333148491559
- https://grist.slack.com/archives/C0234CPPXPA/p1663069291726159

I tested what made many `AddOrUpdateRecord` actions slow in the first place. It was almost entirely due to producing many individual `AddRecord` user actions. About half of that time was for processing the resulting `AddRecord` doc actions. Lookups and updates were not a problem. With these changes, the slowness is gone.

The Python user action implementation is more complex but there are no surprises. The JS API now groups `records` based on the keys of `require` and `fields` so that `BulkAddOrUpdateRecord` can be applied to each group.

Test Plan: Update and extend Python and DocApi tests.

Reviewers: jarek, paulfitz

Reviewed By: jarek, paulfitz

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3642
2022-09-28 17:58:33 +02:00
Alex Hall
d140b49ba3 (core) Include helper columns in ACL rules
Summary: Extend the way ACL resources are read in the server so that if a rule applies to a specific column then that rule also applies to helper columns belonging to that column, as well as helper columns belonging to fields which display that column. This is particularly intended for display columns of reference columns, but it also applies to conditional formatting rule columns.

Test Plan: Added a server test

Reviewers: paulfitz, jarek

Reviewed By: paulfitz, jarek

Differential Revision: https://phab.getgrist.com/D3628
2022-09-26 16:08:56 +02:00
Paul Fitzpatrick
d55b5110ac (core) remove deprecated /download endpoint in favor of newer /api/docs/NNNN/download
Summary:
This endpoint has started to fail when called between a pair
of doc workers. The simplest fix is to simply remove it, it serves no
purpose.

Test Plan: added basic deployment test

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3636
2022-09-20 15:26:04 -04:00
Louis Delbosc
494a683332
Export xlsx #256 (#270)
XLSX export of active view / table

Co-authored-by: Louis Delbosc <louis.delbosc.prestataire@anct.gouv.fr>
Co-authored-by: Vincent Viers <vincent.viers@beta.gouv.fr>
2022-09-14 14:55:44 -04:00
George Gevoian
965cf7e076 Fix incorrect CSS fallback values 2022-09-08 14:30:46 -07:00
George Gevoian
ec157dc469 (core) Add dark mode to user preferences
Summary:
Adds initial implementation of dark mode. Preferences for dark mode are
available on the account settings page. Dark mode is currently a beta feature
as there are still some small bugs to squash and a few remaining UI elements
to style.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: paulfitz, jarek

Differential Revision: https://phab.getgrist.com/D3587
2022-09-05 19:17:32 -07:00
Dmitry S
1c24bfc8a6 (core) Fix exports to CSV/XLSX/etc when data is restricted by access rules
Summary:
- The issue manifested as error "Cannot read property '0' of undefined" in some
  cases, and as "Blocked by table read access rules" in others (instead of
  limiting output to what's not blocked)
- Goes deeper: exports weren't respecting metadata censoring.
- The fix changes exports to use censored metadata, which addresses both errors above.
- Includes an improvement to column ordering in XLSX exports.

Test Plan: Add a server test for CSV and XLSX exports with access rules

Reviewers: paulfitz, georgegevoian

Reviewed By: paulfitz, georgegevoian

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3615
2022-09-02 10:59:59 -04:00
George Gevoian
2cb783ea7b (core) Fix bugs with intervals
Summary:
Fixes some bugs involving intervals, and updates RandomizedTimer to support both fixed and
randomized delays, and to better handle async callbacks.

 * Fixed a bug where Throttle would queue up many pidusage calls due to the use of
    setInterval, and the async nature of the calls.

 * Fixed a but where RandomizedTimer (now just Interval) would not be disabled in
    ActiveDoc on doc shutdown if initialization had not yet settled.

Test Plan: Tested manually.

Reviewers: jarek, dsagal

Reviewed By: jarek, dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3604
2022-08-25 12:38:36 -07:00
Jarosław Sadziński
2997434815 (core) Showing a raw data section on a popup
Summary:
Show raw data will now open a popup with
raw section instead of redirecting to raw data page.

Adding new anchor link type "a2" that is able to open
any section in a popup on a current view.

Not related:
Fixing highlightMatches function, after merging core PR.

Test Plan: Updated tests

Reviewers: alexmojaki, georgegevoian

Reviewed By: alexmojaki, georgegevoian

Subscribers: georgegevoian, alexmojaki

Differential Revision: https://phab.getgrist.com/D3592
2022-08-25 09:30:07 +02:00
George Gevoian
56e8e1f4b3 (core) Randomize ActiveDoc interval delays
Summary:
When large numbers of documents were restarted simultaneously, they had
a tendency to schedule intervals to begin at roughly the same moment in
time, causing periodic spikes in load. This randomizes the delay of each
interval to help avoid such spikes.

Test Plan: Tested manually.

Reviewers: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3600
2022-08-23 23:08:07 -07:00
George Gevoian
ed37401b2c (core) Add basic activation page to grist-ee
Summary:
Adds an activation page to grist-ee that currently shows activation status.

Follow-up diffs will introduce additional functionality, such as the ability to
enter activation keys directly from the activation page.

Test Plan: No grist-ee tests (yet).

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: dsagal, paulfitz

Differential Revision: https://phab.getgrist.com/D3582
2022-08-23 10:30:52 -07:00
Jarosław Sadziński
a52d56f613 (core) Moving client and common tests to core
Summary:
- Moved /test/client and /test/common to core.
- Moved two files (CircularArray and RecentItems) from app/common to core/app/common.
- Moved resetOrg test to gen-server.
- `testrun.sh` is now invoking common and client test from core.
- Added missing packages to core's package.json (and revealed underscore as it is used in the main app).
- Removed Coord.js as it is not used anywhere.

Test Plan: Existing tests

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3590
2022-08-23 19:20:10 +02:00
Jarosław Sadziński
9e4d802405 (core) Implementing row conditional formatting
Summary:
Conditional formatting can now be used for whole rows.
Related fix:
- Font styles weren't applicable for summary columns.
- Checkbox and slider weren't using colors properly

Test Plan: Existing and new tests

Reviewers: paulfitz, georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3547
2022-08-09 20:11:36 +02:00
Jarosław Sadziński
ee109e9186 (core) Removing NEW_DEAL flag
Summary:
Removing NEW_DEAL flag checks and exposing all new deal features as default.
Also removing Pages.ts as it was moved to grist-core.

Test Plan: Existing and updated tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3572
2022-08-09 17:29:28 +02:00
George Gevoian
771e1edd54 (core) Keep track of row counts per table
Summary: Displays a live row count of each table on the Raw Data page.

Test Plan: Browser tests.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3540
2022-08-03 08:13:33 -07:00
Jarosław Sadziński
40c9b8b7e8 (core) New URL that opens Create site popup.
Summary: Adding new url parameter for team site creation

Test Plan: Updated tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3554
2022-08-03 13:09:18 +02:00
George Gevoian
aeba738f7c (core) Add product for new personal plan
Summary:
Adds the new personal plan as a product that will be available
in the future. Can be enabled along with other plan-related via
an environment variable.

Test Plan: Browser tests and existing tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3533
2022-07-26 11:33:23 -07:00
Jarosław Sadziński
5c8211c61d (core) Fixing selectby error on the creator panel.
Summary: [Select By] in the creator panel was bugged. It wasn't refreshed in some cases as the observable array that needed to be created seemed too complicated. This Diff recomputes this array when the user wants to change the selection.

Test Plan: added tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3541
2022-07-26 15:46:11 +02:00
Alex Hall
460f22b701 (core) Don't show hidden tables in table data options when adding widgets
Summary:
As reported in https://grist.slack.com/archives/C069RUP71/p1655316194602829, when a table is hidden by ACL, it was still showing as a blank 'ghost' option to select data when adding a widget:

{F55498}

The fix is simply to return `true` from `isHiddenTable` for empty table IDs, which indicate a table hidden by ACL. `TableRec.isHidden` is supposed to match this so I updated it too, and I cleaned up a tiny bit of other related code.

Test Plan: Extended `nbrowser/AccessRules1.ts` to test the data options when adding widgets.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3530
2022-07-21 16:28:28 +02:00
Paul Fitzpatrick
dd8d2e18f5 (core) add an access token mechanism to help with attachments in custom widgets
Summary:
With this, a custom widget can render an attachment by doing:
```
const tokenInfo = await grist.docApi.getAccessToken({readOnly: true});
const img = document.getElementById('the_image');
const id = record.C[0];  // get an id of an attachment
const src = `${tokenInfo.baseUrl}/attachments/${id}/download?auth=${tokenInfo.token}`;
img.setAttribute('src', src)
```

The access token expires after a few mins, so if a user right-clicks on an image
to save it, they may get access denied unless they refresh the page. A little awkward,
but s3 pre-authorized links behave similarly and it generally isn't a deal-breaker.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3488
2022-07-19 11:55:18 -04:00
Alex Hall
f39b496563 (core) Use table title instead of ID in ACL UI
Summary:
Use table titles (i.e. the raw data widget titles) in dropdowns and other parts of the Acess Rules page, instead of the table ID. This is particularly meant for summary tables which have/had an ID of the form `GristSummary_SourceTable_N`, but https://phab.getgrist.com/D3508 is changing that anyway.

The server method `getAclResources` now returns more metadata about each table so that the UI can display titles.

Test Plan: Extended and updated `nbrowser/AccessRules2.ts`. Added a small unit test for constructing table titles from the new description returned by `getAclResources`.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3494
2022-07-19 16:27:17 +02:00
Jarosław Sadziński
257dafe423 (core) Improving billing page user experience
Summary:
Improving billing page user experience.
- Updated labels for canceled plan
- Adding option to downgrade from team plan to free team plan
- Updating default name for teamFree plan when it is not available in Stripe
- Minor bug fixes

Test Plan: updated tests

Reviewers: cyprien

Reviewed By: cyprien

Subscribers: cyprien

Differential Revision: https://phab.getgrist.com/D3515
2022-07-18 14:43:14 +02:00
Alex Hall
b8486dcdba (core) Nice summary table IDs
Summary:
Changes auto-generated summary table IDs from e.g. `GristSummary_6_Table1` to `Table1_summary_A_B` (meaning `Table1` grouped by `A` and `B`). This makes it easier to write formulas involving summary tables, make API requests, understand logs, etc.

Because these don't encode the source table ID as reliably as before, `decode_summary_table_name` now uses the summary table schema info, not just the summary table ID. Specifically, it looks at the type of the `group` column, which is `RefList:<source table id>`.

Renaming a source table renames the summary table as before, and now renaming a groupby column renames the summary table as well.

Conflicting table names are resolved in the usual way by adding a number at the end, e.g. `Table1_summary_A_B2`. These summary tables are not automatically renamed when the disambiguation is no longer needed.

A new migration renames all summary tables to the new scheme, and updates formulas using summary tables with a simple regex.

Test Plan:
Updated many tests to use the new style of name.

Added new Python tests to for resolving conflicts when renaming source tables and groupby columns.

Added a test for the migration, including renames in formulas.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3508
2022-07-14 12:09:56 +02:00
Paul Fitzpatrick
f91f45b26d (core) support granular read access for attachments
Summary:
When a user requests to read the contents of an attachment, only allow the request if there exists a cell in an attachment column that contains the attachment and which they have read access to.

This does not cover:
 * Granular write access for attachments. In particular, a user who can write to any attachment column should be considered to have full read access to all attachment columns, currently.
 * Access control of attachment metadata such as name and format.

The implementation uses a sql query that requires a scan, and some notes on how this could be optimized in future. The web client was updated to specify the cell to check for access, and performance seemed fine in casual testing on a doc with 1000s of attachments. I'm not sure how performance would hold up as the set of access rules grows as well.

Test Plan: added tests

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3490
2022-07-07 07:22:02 -04:00
George Gevoian
a051830aeb (core) Show summary tables on Raw Data page
Summary:
Summary tables now have their own raw viewsection, and are shown
under Raw Data Tables on the Raw Data page.

Test Plan: Browser and Python tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3495
2022-07-06 09:41:48 -07:00
Dmitry S
51ff72c15e (core) Faster builds all around.
Summary:
Building:
- Builds no longer wait for tsc for either client, server, or test targets. All use esbuild which is very fast.
- Build still runs tsc, but only to report errors. This may be turned off with `SKIP_TSC=1` env var.
- Grist-core continues to build using tsc.
- Esbuild requires ES6 module semantics. Typescript's esModuleInterop is turned
  on, so that tsc accepts and enforces correct usage.
- Client-side code is watched and bundled by webpack as before (using esbuild-loader)

Code changes:
- Imports must now follow ES6 semantics: `import * as X from ...` produces a
  module object; to import functions or class instances, use `import X from ...`.
- Everything is now built with isolatedModules flag. Some exports were updated for it.

Packages:
- Upgraded browserify dependency, and related packages (used for the distribution-building step).
- Building the distribution now uses esbuild's minification. babel-minify is no longer used.

Test Plan: Should have no behavior changes, existing tests should pass, and docker image should build too.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3506
2022-07-04 10:42:40 -04:00
Jarosław Sadziński
aefe451bab (core) Polishing upgrade plan UI
Summary:
- Update nudge boxes content and collapsing on personal and free team site
- New confirmation after upgrading from a free team site
- Refactoring ProductUpgrade code, splitting plans / modals and nudges

Test Plan: Manual and updated tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3481
2022-06-29 18:28:37 +02:00
Dmitry S
dd2eadc86e (core) Speed up and upgrade build.
Summary:
- Upgrades to build-related packages:
  - Upgrade typescript, related libraries and typings.
  - Upgrade webpack, eslint; add tsc-watch, node-dev, eslint_d.

- Build organization changes:
  - Build webpack from original typescript, transpiling only; with errors still
    reported by a background tsc watching process.

- Typescript-related changes:
  - Reduce imports of AWS dependencies (very noticeable speedup)
  - Avoid auto-loading global @types
  - Client code is now built with isolatedModules flag (for safe transpilation)
  - Use allowJs to avoid copying JS files manually.

- Linting changes
  - Enhance Arcanist ESLintLinter to run before/after commands, and set up to use eslint_d
  - Update eslint config, and include .eslintignore to avoid linting generated files.
  - Include a bunch of eslint-prompted and eslint-generated fixes
  - Add no-unused-expression rule to eslint, and fix a few warnings about it

- Other items:
  - Refactor cssInput to avoid circular dependency
  - Remove a bit of unused code, libraries, dependencies

Test Plan: No behavior changes, all existing tests pass. There are 30 tests fewer reported because `test_gpath.py` was removed (it's been unused for years)

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3498
2022-06-27 16:10:10 -04:00
Cyprien P
64ff9ccd0a (core) Allows range filter for Date, DateTime columns
Summary:
This diff is first of a series of 3 commits to enable range filering
for Date and DateTime columns. Diff only enable setting date's min/max
throw typing dates, Date picker and relative ranges are left for
follow-up commits.

 - Exposes columns value formatter to the range input
 - Fixes column filter func to work with dates

Test Plan:
Adds Date to projects range filter test
Adds Date/DateTime to nbrowser ColumnFilterMenu tests

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3455
2022-06-23 13:02:37 +02:00
Alex Hall
9fffb491f9 (core) External requests
Summary:
Adds a Python function `REQUEST` which makes an HTTP GET request. Behind the scenes it:

- Raises a special exception to stop trying to evaluate the current cell and just keep the existing value.
- Notes the request arguments which will be returned by `apply_user_actions`.
- Makes the actual request in NodeJS, which sends back the raw response data in a new action `RespondToRequests` which reevaluates the cell(s) that made the request.
- Wraps the response data in a class which mimics the `Response` class of the `requests` library.

In certain cases, this asynchronous flow doesn't work and the sandbox will instead synchronously call an exported JS method:

- When reevaluating a single cell to get a formula error, the request is made synchronously.
- When a formula makes multiple requests, the earlier responses are retrieved synchronously from files which store responses as long as needed to complete evaluating formulas. See https://grist.slack.com/archives/CL1LQ8AT0/p1653399747810139

Test Plan: Added Python and nbrowser tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: paulfitz, dsagal

Differential Revision: https://phab.getgrist.com/D3429
2022-06-17 21:53:20 +02:00
George Gevoian
abebe812db (core) Show Grist video tour after welcome questions
Summary:
After the welcome questions are dismissed, a video tour modal will
now be displayed. The video tour is also accessible via a tool button
in the left panel of the home page, as well as a text button next to
the Examples & Templates header.

Test Plan: Browser tests.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3477
2022-06-16 21:54:17 -07:00
Dmitry S
a91d493ffc (core) Fix issue with 'UNEXPECTED ORDER OF CALLBACKS' in Client.ts.
Summary:
- Substantial refactoring of the logic when the server fails to send some
  messages to a client.
- Add seqId numbers to server messages to ensure reliable order.
- Add a needReload flag in clientConnect for a clear indication whent the
  browser client needs to reload the app.
- Reproduce some potential failure scenarios in a test case (some of which
  previously could have led to incorrectly ordered messages).
- Convert other Comm tests to typescript.
- Tweak logging of Comm and Client to be slightly more concise (in particular,
  avoid logging sessionId)

Note that despite the big refactoring, this only addresses a fairly rare
situation, with websocket failures while server is trying to send to the
client. It includes no improvements for failures while the client is sending to
the server.

(I looked for an existing library that would take care of these issues. A relevant article I found is https://docs.microsoft.com/en-us/azure/azure-web-pubsub/howto-develop-reliable-clients, but it doesn't include a library for both ends, and is still in review. Other libraries with similar purposes did not inspire enough confidence.)

Test Plan: New test cases, which reproduce some previously problematic scenarios.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3470
2022-06-16 23:51:14 -04:00
Paul Fitzpatrick
561d9696aa (core) clean up interaction of forward auth with session
Summary:
For self-hosted Grist, forward auth has proven useful, where
some proxy wrapped around Grist manages authentication, and
passes on user information to Grist in a trusted header.
The current implementation is adequate when Grist is the
only place where the user logs in or out, but is confusing
otherwise (see https://github.com/gristlabs/grist-core/issues/207).
Here we take some steps to broaden the scenarios Grist's
forward auth support can be used with:

  * When a trusted header is present and is blank, treat
    that as the user not being logged in, and don't look
    any further for identity information. Specifically,
    don't look in Grist's session information.
  * Add a `GRIST_IGNORE_SESSION` flag to entirely prevent
    Grist from picking up identity information from a cookie,
    in order to avoid confusion between multiple login methods.
  * Add tests for common scenarios.

Test Plan: added tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3482
2022-06-15 13:06:12 -04:00
Alex Hall
0350e2df58 (core) Fix filtering of empty reflists
Summary:
A formula returning an empty RecordSet in a RefList columns results in storing [] instead of null.
This caused a bug where the empty list was 'flattened' and the cell not appearing in filters at all.
This diff fixes the bug by filtering for the default value `null` instead for RefLists and the empty string for ChoiceLists.
I didn't manage to actually reproduce the bug for ChoiceLists, but this seemed the most sensible thing to do.

Test Plan: New nbrowser test.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3478
2022-06-11 00:16:57 +02:00
Alex Hall
1c89d08ea3 (core) Add a row to summary tables grouped by list column(s) corresponding to empty lists
Summary:
Adds some special handling to summary table and lookup logic:

- Source rows with empty choicelists/reflists get a corresponding summary row with an empty string/reference when grouping by that column, instead of excluding them from any group
- Adds a new `QueryOperation` 'empty' in the client which is used in `LinkingState`, `QuerySet`, and `recursiveMoveToCursorPos` to match empty lists in source tables against falsy values in linked summary tables.
- Adds a new parameter `match_empty` to the Python `CONTAINS` function so that regular formulas can implement the same behaviour as summary tables. See https://grist.slack.com/archives/C0234CPPXPA/p1654030490932119
- Uses the new `match_empty` argument in the formula generated for the `group` column when detaching a summary table.

Test Plan: Updated and extended Python and nbrowser tests of summary tables grouped by choicelists to test for new behaviour with empty lists.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3471
2022-06-09 23:38:14 +02:00
Jarosław Sadziński
d92a761f6e (core) Product update popups and hosted stripe integration
Summary:
- Showing nudge to individual users to sign up for free team plan.
- Implementing billing page to upgrade from free team to pro.
- New modal with upgrade options and free team site signup.
- Integrating Stripe-hosted UI for checkout and plan management.

Test Plan: updated tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3456
2022-06-08 21:10:49 +02:00
Dmitry S
4f1cb53b29 (core) Converting server-side Comm.js to typescript
Summary:
- Add app/common/CommTypes.ts to define types shared by client and server.
- Include @types/ws npm package

Test Plan: Intended to have no changes in behavior

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3467
2022-06-07 15:47:17 -04:00
Paul Fitzpatrick
05d1cdf140 (core) limit retries of uploads to external store in tests
Summary:
If an external store fails completely, Grist will continue to
retry uploading to it. This diff updated the HostedStorageManager
test to limit the extent of these retries to the test itself -
otherwise they continue for all other tests in the same process,
potentially disrupting those that read logs. There are other tests
that use s3, but they aren't run in the same process with delicate
log-reading tests, and it isn't quite as clear what improvement
to make there.

Test Plan:
artificially made external store fail, and checked that
test contamination seen previously no longer occurs.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3469
2022-06-06 16:19:41 -04:00
George Gevoian
6dcdd22792 (core) Redirect less often in welcomeNewUser
Summary:
Instead of always redirecting new users to the home page or the (teams) welcome page,
only redirect when the user signed in for the first time on a personal site, has access to
other sites, and isn't already being redirected to a specific page on their personal site.

Also tweaks how invalid Choice column values are displayed to match Choice List
columns, and fixes a small CSS issue with select by in the page widget picker when
there are options with long labels.

Test Plan: Browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3461
2022-06-06 11:26:49 -07:00
George Gevoian
090d9af21d (core) Broadcast doc usage updates to clients
Summary:
Introduces a new message type, docUsage, that's broadcast to all connected
clients whenever document usage is updated in ActiveDoc.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3451
2022-06-06 09:55:34 -07:00
Dmitry S
acddd25cfd (core) Update design of empty docs home page, and add a "Manage Team" button.
Summary:
- Remove the empty-folder icon
- Add an "Invite team members" button for owners on empty team sites
- Add a "Browse Templates" button for all other cases on empty sites
- Update intro text for team, including a link to Sprouts
- Update intro text for personal/anon.
- Include a Free/Pro tag for team sites (for now, only "Free")
- Add a "Manage Team" button for owners on home page of all team sites.
- Polished the UI of UserManager: add a transition for the background, and
  delay the appearance of the spinner for fast loads.

Test Plan: Fixed up the HomeIntro tests; added test case for Manage Team button

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3459
2022-06-03 12:58:57 -04:00
Cyprien P
815c9e1462 (core) Adds new range filter for numeric columns
Summary:
Shows the range filter next to the filter by values on filter menu. When users
set min and/or max, it takes precendence over the filter by values.

If users set:
 - `[] < [max]` behaves as `less than max`.
 - `[min] < []` behaves as `more than min`.
 - `[min] < [max]` behaves as `between min and max`
 - bounds are always inclusives.
 - when users change min or max the values of the by values filter
   gets checked/unchecked depending on whether they are included by
   the range filter.
 - when users clicks any btn/checkbox of the by values filter both min
   and max input gets cleared, and the filter convert to a filter by
   values.

Test Plan: Adds both projets and nbrowser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3435
2022-05-30 10:30:16 +02:00
Alex Hall
6b372fa6cd (core) Allow configuring (mostly hiding) various little bits of UI
Summary:
Adds two new env vars GRIST_HIDE_UI_ELEMENTS and GRIST_PAGE_TITLE_SUFFIX which translate to values in GristLoadConfig that the server sends the client when loading.

For checkin task https://gristlabs.getgrist.com/doc/check-ins/p/5#a1.s9.r1882.c19

Test Plan: Tested manually

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3449
2022-05-27 14:32:05 +02:00
George Gevoian
74ec9358da (core) Show usage banners in doc menu of free team sites
Summary:
Also fixes a minor CSS regression in UserManager where the
link to add a team member wasn't shown on a separate row.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3444
2022-05-26 15:01:35 -07:00
Dmitry S
309ddb0fe7 (core) Move guessing logic for column types to run in node once for all columns.
Summary:
Previously, columns of type Any were created and modified one by one by reusing
the "empty column" logic from the data engine. This copies that logic to Node,
and sets the type of all columns together, to create them with the correct type
in the AddTable call.

This makes imports about twice faster (when slowness is due to many columns),
but doesn't address all cases where individual handling of columns causes slowness.

Test Plan: Added a test case for the new helper function.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3427
2022-05-19 12:49:51 -04:00
George Gevoian
a6063f570a (core) Polish Access Details
Summary:
Instead of showing a blank dialog for users whose access
is limited (e.g. public members), we now show the user's
role and a mention of whether their access is public.

Test Plan: Browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3431
2022-05-19 09:45:26 -07:00
Jarosław Sadziński
0ab9e4a6a0 (core) Adding GristConnect login system
Summary:
New login system to allow simple SSO flow that is based on Discourse description that is available at:
https://meta.discourse.org/t/discourseconnect-official-single-sign-on-for-discourse-sso/13045

Test Plan: New core test.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3418
2022-05-18 20:28:25 +02:00
George Gevoian
2fd8a34ff8 (core) Move Notifier to /ext
Summary:
This makes it possible to configure a SendGrid-based Notifier
instance via a JSON configuration file.

Test Plan: Tested manually.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3432
2022-05-18 08:02:32 -07:00
Paul Fitzpatrick
e4d47a2f3c (core) add minimal support for activation keys
Summary: For grist-ee, expect an activation key in environment variable `GRIST_ACTIVATION` or in a file pointed to by `GRIST_ACTIVATION_FILE`. In absence of key, start a 30-day trial, during which a banner is shown. Once trial expires, installation goes into document-read-only mode.

Test Plan: added a test

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3426
2022-05-16 22:46:23 -04:00
George Gevoian
f48d579f64 (core) Add API endpoint to get site usage summary
Summary:
The summary includes a count of documents that are approaching
limits, in grace period, or delete-only. The endpoint is only accessible
to site owners, and is currently unused. A follow-up diff will add usage
banners to the site home page, which will use the response from the
endpoint to communicate usage information to owners.

Test Plan: Browser and server tests.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3420
2022-05-16 11:16:19 -07:00
Alex Hall
4408315f2e (core) Add AzureExternalStorage
Summary:
Adds a new implementation of the interface ExternalStorage that works for Azure Blob Storage as an alternative to S3, for a specific self-hosting case.

Tweaks HostedStorageManager and ICreate to allow configuring different core implementations of ExternalStorage.

Followup tasks:

- Make this code available to self hosters, possibly by making it open source.
- Add an env var or other config option to specify the preferred type of storage. Currently using the var `AZURE_STORAGE_CONNECTION_STRING` to know how to connect to Azure when requested, but that choice still only lives in test code.

Test Plan: Generalized HostedStorageManager and ExternalStorage tests to test the new AzureExternalStorage alongside S3ExternalStorage. The HostedStorageManager tests also now test the 'cached' in-memory test storage in a way that's closer to the real storage methods.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3413
2022-05-09 21:44:57 +02:00
George Gevoian
1e42871cc9 (core) Add attachment and data size usage
Summary:
Adds attachment and data size to the usage section of
the raw data page. Also makes in-document usage banners
update as user actions are applied, causing them to be
hidden/shown or updated based on the current state of
the document.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3395
2022-05-04 13:46:55 -07:00
Jarosław Sadziński
6f00106d7c (core) Raw renames
Summary:
A new way for renaming tables.

  - There is a new popup to rename section (where you can also rename the table)
  - Renaming/Deleting page doesn't modify/delete the table.
  - Renaming table can rename a page if the names match (and the page contains a section with that table).
  - User can rename table in Raw Data UI in two ways - either on the listing or by using the section name popup
  - As before, there is no way to change tableId - it is derived from a table name.
  - When the section name is empty the table name is shown instead.
  - White space for section name is allowed (to discuss) - so the user can just paste '   '.
  - Empty name for a page is not allowed (but white space is).
  - Some bugs related to deleting tables with attached summary tables (and with undoing this operation) were fixed (but not all of them yet).

Test Plan: Updated tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: georgegevoian

Differential Revision: https://phab.getgrist.com/D3360
2022-04-27 22:21:55 +02:00
Jarosław Sadziński
995bf9b63a (core) Distinct style rules for summary columns
Summary:
Summary columns now have their own conditional rules,
which are not shared with sister columns.

Test Plan: New test

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3388
2022-04-27 20:51:23 +02:00
George Gevoian
af5b3c9004 (core) Add document usage banners
Summary:
This also enables the new Usage section for all sites. Currently,
it shows metrics for document row count, but only if the user
has full document read access. Otherwise, a message about
insufficient access is shown.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3377
2022-04-25 08:14:52 -07:00
Alex Hall
64a5c79dbc (core) Limit total attachment file size per document
Summary:
- Add a new parameter `Features.baseMaxAttachmentsBytesPerDocument` and set it to 1GB for the free team product.
- Add a method to DocStorage to calculate the total size of existing and used attachments.
- Add a migration to DocStorage adding an index to make the query in the above method fast.
- Check in ActiveDoc if uploading attachment(s) would exceed the product limit on that document.

Test Plan: Added test in `limits.ts` testing enforcement of the attachment limit.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3374
2022-04-14 16:33:09 +02:00
Paul Fitzpatrick
20dd2fc70d (core) allow non-owners to remove themselves from sites/workspaces/docs
Summary:
For users who cannot otherwise change access to a resource, let
them remove themselves. Implemented via the standard endpoints
as a special exception that will process a request from a user
that would otherwise be denied, if the only contents of that
request are a removal of themselves.

Users who can change access are still not permitted to change their
own permissions or to remove themselves, as a precaution against
orphaning resources.

Test Plan: extended and updated tests

Reviewers: cyprien

Reviewed By: cyprien

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3367
2022-04-13 10:04:32 -04:00
George Gevoian
859c593448 (core) Add authSubject and authProvider to sessions
Summary:
This also updates Authorizer to link the authSubject
to Grist users if not previously linked. Linked subjects
are now used as the username for password-based logins,
instead of emails, which remain as a fallback.

Test Plan: Existing tests, and tested login flows manually.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3356
2022-04-11 11:42:02 -07:00
Paul Fitzpatrick
14f7e30e6f (core) add users.options.isConsultant flag, and omit such users from billing
Summary:
This adds an optional `isConsultant` flag to `users.options`, and an endpoint that allows the support user to turn it on or off. Users marked as consultants are not counted as billable members. Follows the example of existing `allowGoogleLogin` option.

Billable members are counted when members are added or removed from a site. Changing the `isConsultant` flag has no immediate or retroactive effect on billing. The number of users in stripe is now set unconditionally, rather than only when it has changed.

Notifications to billing managers are not aware of this billing nuance, but continue to report user counts that include consultants. The notifications link users to the billing page.

Test Plan: extended test

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: anaisconce, jarek

Differential Revision: https://phab.getgrist.com/D3362
2022-04-11 10:26:31 -04:00
Jarosław Sadziński
34708cd348 (core) Adding font options to the style picker
Summary:
Redesigning color picker:
- Single color palette (no light/dark switch)
- Ability to remove color (new empty button)

New font options in the color picker.
Font options are available on:
- Default cell style
- Conditional rules styles
- Choice/ChoiceList editor and token field
- Filters for Choice/ChoiceList columns

Design document:
https://www.figma.com/file/bRTsb47VIOVBfJPj0qF3C9/Grist-Updates?node-id=415%3A8135

Test Plan: new and updated tests

Reviewers: georgegevoian, alexmojaki

Reviewed By: georgegevoian, alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3335
2022-04-07 20:35:03 +02:00
Alex Hall
251d79704b (core) Migrate Attachments columns from marshalled blobs to JSON
Summary: Adds a migration in preparation for future work on tracking and deleting attachments. This includes a `_grist_Attachments.timeDeleted` column which isn't used yet, and changing the storage format of user columns of type `Attachments`. DocStorage now treats Attachments like RefList in general (since they use JSON), which also prompted a tiny bit of refactoring.

Test Plan: Added a migration test case showing the change in format.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3352
2022-04-06 13:28:47 +02:00