Commit Graph

222 Commits

Author SHA1 Message Date
Dmitry S
acddd25cfd (core) Update design of empty docs home page, and add a "Manage Team" button.
Summary:
- Remove the empty-folder icon
- Add an "Invite team members" button for owners on empty team sites
- Add a "Browse Templates" button for all other cases on empty sites
- Update intro text for team, including a link to Sprouts
- Update intro text for personal/anon.
- Include a Free/Pro tag for team sites (for now, only "Free")
- Add a "Manage Team" button for owners on home page of all team sites.
- Polished the UI of UserManager: add a transition for the background, and
  delay the appearance of the spinner for fast loads.

Test Plan: Fixed up the HomeIntro tests; added test case for Manage Team button

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3459
2022-06-03 12:58:57 -04:00
Cyprien P
815c9e1462 (core) Adds new range filter for numeric columns
Summary:
Shows the range filter next to the filter by values on filter menu. When users
set min and/or max, it takes precendence over the filter by values.

If users set:
 - `[] < [max]` behaves as `less than max`.
 - `[min] < []` behaves as `more than min`.
 - `[min] < [max]` behaves as `between min and max`
 - bounds are always inclusives.
 - when users change min or max the values of the by values filter
   gets checked/unchecked depending on whether they are included by
   the range filter.
 - when users clicks any btn/checkbox of the by values filter both min
   and max input gets cleared, and the filter convert to a filter by
   values.

Test Plan: Adds both projets and nbrowser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3435
2022-05-30 10:30:16 +02:00
Alex Hall
6b372fa6cd (core) Allow configuring (mostly hiding) various little bits of UI
Summary:
Adds two new env vars GRIST_HIDE_UI_ELEMENTS and GRIST_PAGE_TITLE_SUFFIX which translate to values in GristLoadConfig that the server sends the client when loading.

For checkin task https://gristlabs.getgrist.com/doc/check-ins/p/5#a1.s9.r1882.c19

Test Plan: Tested manually

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3449
2022-05-27 14:32:05 +02:00
George Gevoian
74ec9358da (core) Show usage banners in doc menu of free team sites
Summary:
Also fixes a minor CSS regression in UserManager where the
link to add a team member wasn't shown on a separate row.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3444
2022-05-26 15:01:35 -07:00
Dmitry S
309ddb0fe7 (core) Move guessing logic for column types to run in node once for all columns.
Summary:
Previously, columns of type Any were created and modified one by one by reusing
the "empty column" logic from the data engine. This copies that logic to Node,
and sets the type of all columns together, to create them with the correct type
in the AddTable call.

This makes imports about twice faster (when slowness is due to many columns),
but doesn't address all cases where individual handling of columns causes slowness.

Test Plan: Added a test case for the new helper function.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3427
2022-05-19 12:49:51 -04:00
George Gevoian
a6063f570a (core) Polish Access Details
Summary:
Instead of showing a blank dialog for users whose access
is limited (e.g. public members), we now show the user's
role and a mention of whether their access is public.

Test Plan: Browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3431
2022-05-19 09:45:26 -07:00
Jarosław Sadziński
0ab9e4a6a0 (core) Adding GristConnect login system
Summary:
New login system to allow simple SSO flow that is based on Discourse description that is available at:
https://meta.discourse.org/t/discourseconnect-official-single-sign-on-for-discourse-sso/13045

Test Plan: New core test.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3418
2022-05-18 20:28:25 +02:00
George Gevoian
2fd8a34ff8 (core) Move Notifier to /ext
Summary:
This makes it possible to configure a SendGrid-based Notifier
instance via a JSON configuration file.

Test Plan: Tested manually.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3432
2022-05-18 08:02:32 -07:00
Paul Fitzpatrick
e4d47a2f3c (core) add minimal support for activation keys
Summary: For grist-ee, expect an activation key in environment variable `GRIST_ACTIVATION` or in a file pointed to by `GRIST_ACTIVATION_FILE`. In absence of key, start a 30-day trial, during which a banner is shown. Once trial expires, installation goes into document-read-only mode.

Test Plan: added a test

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3426
2022-05-16 22:46:23 -04:00
George Gevoian
f48d579f64 (core) Add API endpoint to get site usage summary
Summary:
The summary includes a count of documents that are approaching
limits, in grace period, or delete-only. The endpoint is only accessible
to site owners, and is currently unused. A follow-up diff will add usage
banners to the site home page, which will use the response from the
endpoint to communicate usage information to owners.

Test Plan: Browser and server tests.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3420
2022-05-16 11:16:19 -07:00
Alex Hall
4408315f2e (core) Add AzureExternalStorage
Summary:
Adds a new implementation of the interface ExternalStorage that works for Azure Blob Storage as an alternative to S3, for a specific self-hosting case.

Tweaks HostedStorageManager and ICreate to allow configuring different core implementations of ExternalStorage.

Followup tasks:

- Make this code available to self hosters, possibly by making it open source.
- Add an env var or other config option to specify the preferred type of storage. Currently using the var `AZURE_STORAGE_CONNECTION_STRING` to know how to connect to Azure when requested, but that choice still only lives in test code.

Test Plan: Generalized HostedStorageManager and ExternalStorage tests to test the new AzureExternalStorage alongside S3ExternalStorage. The HostedStorageManager tests also now test the 'cached' in-memory test storage in a way that's closer to the real storage methods.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3413
2022-05-09 21:44:57 +02:00
George Gevoian
1e42871cc9 (core) Add attachment and data size usage
Summary:
Adds attachment and data size to the usage section of
the raw data page. Also makes in-document usage banners
update as user actions are applied, causing them to be
hidden/shown or updated based on the current state of
the document.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3395
2022-05-04 13:46:55 -07:00
Jarosław Sadziński
6f00106d7c (core) Raw renames
Summary:
A new way for renaming tables.

  - There is a new popup to rename section (where you can also rename the table)
  - Renaming/Deleting page doesn't modify/delete the table.
  - Renaming table can rename a page if the names match (and the page contains a section with that table).
  - User can rename table in Raw Data UI in two ways - either on the listing or by using the section name popup
  - As before, there is no way to change tableId - it is derived from a table name.
  - When the section name is empty the table name is shown instead.
  - White space for section name is allowed (to discuss) - so the user can just paste '   '.
  - Empty name for a page is not allowed (but white space is).
  - Some bugs related to deleting tables with attached summary tables (and with undoing this operation) were fixed (but not all of them yet).

Test Plan: Updated tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: georgegevoian

Differential Revision: https://phab.getgrist.com/D3360
2022-04-27 22:21:55 +02:00
Jarosław Sadziński
995bf9b63a (core) Distinct style rules for summary columns
Summary:
Summary columns now have their own conditional rules,
which are not shared with sister columns.

Test Plan: New test

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3388
2022-04-27 20:51:23 +02:00
George Gevoian
af5b3c9004 (core) Add document usage banners
Summary:
This also enables the new Usage section for all sites. Currently,
it shows metrics for document row count, but only if the user
has full document read access. Otherwise, a message about
insufficient access is shown.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3377
2022-04-25 08:14:52 -07:00
Alex Hall
64a5c79dbc (core) Limit total attachment file size per document
Summary:
- Add a new parameter `Features.baseMaxAttachmentsBytesPerDocument` and set it to 1GB for the free team product.
- Add a method to DocStorage to calculate the total size of existing and used attachments.
- Add a migration to DocStorage adding an index to make the query in the above method fast.
- Check in ActiveDoc if uploading attachment(s) would exceed the product limit on that document.

Test Plan: Added test in `limits.ts` testing enforcement of the attachment limit.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3374
2022-04-14 16:33:09 +02:00
Paul Fitzpatrick
20dd2fc70d (core) allow non-owners to remove themselves from sites/workspaces/docs
Summary:
For users who cannot otherwise change access to a resource, let
them remove themselves. Implemented via the standard endpoints
as a special exception that will process a request from a user
that would otherwise be denied, if the only contents of that
request are a removal of themselves.

Users who can change access are still not permitted to change their
own permissions or to remove themselves, as a precaution against
orphaning resources.

Test Plan: extended and updated tests

Reviewers: cyprien

Reviewed By: cyprien

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3367
2022-04-13 10:04:32 -04:00
George Gevoian
859c593448 (core) Add authSubject and authProvider to sessions
Summary:
This also updates Authorizer to link the authSubject
to Grist users if not previously linked. Linked subjects
are now used as the username for password-based logins,
instead of emails, which remain as a fallback.

Test Plan: Existing tests, and tested login flows manually.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3356
2022-04-11 11:42:02 -07:00
Paul Fitzpatrick
14f7e30e6f (core) add users.options.isConsultant flag, and omit such users from billing
Summary:
This adds an optional `isConsultant` flag to `users.options`, and an endpoint that allows the support user to turn it on or off. Users marked as consultants are not counted as billable members. Follows the example of existing `allowGoogleLogin` option.

Billable members are counted when members are added or removed from a site. Changing the `isConsultant` flag has no immediate or retroactive effect on billing. The number of users in stripe is now set unconditionally, rather than only when it has changed.

Notifications to billing managers are not aware of this billing nuance, but continue to report user counts that include consultants. The notifications link users to the billing page.

Test Plan: extended test

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: anaisconce, jarek

Differential Revision: https://phab.getgrist.com/D3362
2022-04-11 10:26:31 -04:00
Jarosław Sadziński
34708cd348 (core) Adding font options to the style picker
Summary:
Redesigning color picker:
- Single color palette (no light/dark switch)
- Ability to remove color (new empty button)

New font options in the color picker.
Font options are available on:
- Default cell style
- Conditional rules styles
- Choice/ChoiceList editor and token field
- Filters for Choice/ChoiceList columns

Design document:
https://www.figma.com/file/bRTsb47VIOVBfJPj0qF3C9/Grist-Updates?node-id=415%3A8135

Test Plan: new and updated tests

Reviewers: georgegevoian, alexmojaki

Reviewed By: georgegevoian, alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3335
2022-04-07 20:35:03 +02:00
Alex Hall
251d79704b (core) Migrate Attachments columns from marshalled blobs to JSON
Summary: Adds a migration in preparation for future work on tracking and deleting attachments. This includes a `_grist_Attachments.timeDeleted` column which isn't used yet, and changing the storage format of user columns of type `Attachments`. DocStorage now treats Attachments like RefList in general (since they use JSON), which also prompted a tiny bit of refactoring.

Test Plan: Added a migration test case showing the change in format.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D3352
2022-04-06 13:28:47 +02:00
Paul Fitzpatrick
6c6bfee00e (core) fix redirects for multi-team Grist on a single domain
Summary:
The logic for calculating redirects wasn't quite right for Grist
configured to use a single domain, with teams encoded in the path.
This fixes it.

Test Plan: tested manually with docker compose and /etc/hosts

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3359
2022-04-05 17:27:37 -04:00
George Gevoian
6305811ca6 (core) Add new Grist login page
Summary:
Adds a new Grist login page to the login app, and replaces the
server-side Cognito Google Sign-In flow with Google's own OAuth flow.

Test Plan: Browser and server tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3332
2022-04-01 15:24:19 -07:00
Alex Hall
21b0ac3eff (core) Enforcing data size limit
Summary:
Track 'data size' in ActiveDoc alongside row count. Measure it at most once every 5 minutes after each change as before, or after every change when it becomes high enough to matter.

A document is now considered to be approaching/exceeding 'the data limit' if either the data size or the row count is approaching/exceeding its own limit.

Unrelated: tweaked teamFreeFeatures.snapshotWindow based on Quip comments

Test Plan: Tested manually that data size is now logged after every change once it gets high enough, but only if the row limit isn't also too high. Still too early for automated tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3341
2022-03-30 17:56:05 +02:00
Alex Hall
06956f84a5 (core) Make Attachments columns get treated like RefLists more
Summary:
Treat the column type 'Attachments' as equivalent to 'RefList:_grist_Attachments' in a few places, because that's essentially what it is. The main goal was to fix parsing strings representing attachments (reflists).

Also removed an unused function.

Test Plan: Tested manually that pasting a CSV/JSON string representation of an attachments reflists works now.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3338
2022-03-28 23:14:29 +02:00
Alex Hall
59436d2bca (core) Grace period and delete-only mode when exceeding row limit
Summary:
Builds upon https://phab.getgrist.com/D3328

- Add HomeDB column `Document.gracePeriodStart`
- When the row count moves above the limit, set it to the current date. When it moves below, set it to null.
- Add DataLimitStatus type indicating if the document is approaching the limit, is in a grace period, or is in delete only mode if the grace period started at least 14 days ago. Compute it in ActiveDoc and send it to client when opening.
- Only allow certain user actions when in delete-only mode.

Follow-up tasks related to this diff:

- When DataLimitStatus in the client is non-empty, show a banner to the appropriate users.
- Only send DataLimitStatus to users with the appropriate access. There's no risk landing this now since real users will only see null until free team sites are released.
- Update DataLimitStatus immediately in the client when it changes, e.g. when user actions are applied or the product is changed. Right now it's only sent when the document loads.
- Update row limit, grace period start, and data limit status in ActiveDoc when the product changes, i.e. the user upgrades/downgrades.
- Account for data size when computing data limit status, not just row counts.

See also the tasks mentioned in https://phab.getgrist.com/D3331

Test Plan: Extended FreeTeam nbrowser test, testing the 4 statuses.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3331
2022-03-25 13:41:33 +02:00
Jarosław Sadziński
b1c3943bf4 (core) Conditional formatting rules
Summary:
Adding conditional formatting rules feature.

Each column can have multiple styling rules which are applied in order
when evaluated to a truthy value.

- The creator panel has a new section: Cell Style
- New user action AddEmptyRule for adding an empty rule
- New columns in _grist_Table_columns and fields

A new color picker will be introduced in a follow-up diff (as it is also
used in choice/choice list/filters).

Design document:
https://grist.quip.com/FVzfAgoO5xOF/Conditional-Formatting-Implementation-Design

Test Plan: new tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3282
2022-03-23 13:15:02 +01:00
Jarosław Sadziński
96a34122a5 (core) Restoring cursor position on raw data views
Summary:
This diff introduces cursor features for raw data views:
- Restoring cursor position when the browser window is reloaded
- Restoring the last edit position when the browser window is reloaded

Test Plan: Added tests

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3314
2022-03-23 12:24:18 +01:00
Alex Hall
ec8460b772 (core) Prune snapshots outside the window in product features
Summary:
- Add a method `getSnapshotWindow` to `IInventory` and `DocSnapshotInventory`. It returns a `SnapshotWindow`, which represents a duration of time for which we keep backups for a particular document.
- `DocSnapshotPruner` calls this method and passes the window to `shouldKeepSnapshots` to determine which document versions have fallen outside the window and should be pruned.
- The implementation passed to `DocSnapshotInventory` uses a new method `getDocProduct` in `HomeDBManager` which directly returns the `Product` associated with a document, given only the document ID. Other methods in `HomeDBManager` require passing more information, especially about a user, but `DocSnapshotPruner` only knows about document IDs.

Test Plan: Added a test for `getDocProduct` and a test for `DocSnapshotPruner` where `getSnapshotWindow` is specified.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3322
2022-03-18 18:48:14 +02:00
George Gevoian
0f4f0d3dad (core) Migrate to SRP and add change password dialog
Summary:
Moves some auth-related UI components, like MFAConfig, out
of core, and adds a new ChangePasswordDialog component for
allowing direct password changes, replacing the old reset password
link to hosted Cognito.

Updates all MFA endpoints to use SRP for authentication.

Also refactors MFAConfig into smaller files, and polishes up some parts
of the UI to be more consistent with the login pages.

Test Plan: New server and deployment tests. Updated existing tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3311
2022-03-16 21:35:06 -07:00
Paul Fitzpatrick
98f64a8461 (core) add grist.selectedTable.create/update/destroy/upsert to custom widget api
Summary: This makes an equivalent of the /records REST endpoint available within custom widgets. For simple operations, it is compatible with https://github.com/airtable/airtable.js/. About half of the diff is refactoring code from DocApi that implements /records using applyUserActions, to make that code available in the plugin api.

Test Plan: added tests

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3320
2022-03-15 11:11:58 -04:00
Alex Hall
02e69fb685 (core) Crudely show row count and limit in UI
Summary:
Add rowCount returned from sandbox when applying user actions to ActionGroup which is broadcast to clients.

Add rowCount to ActiveDoc and update it after applying user actions.

Add rowCount to OpenLocalDocResult using ActiveDoc value, to show when a client opens a doc before any user actions happen.

Add rowCount observable to DocPageModel which is set when the doc is opened and when action groups are received.

Add crude UI (commented out) in Tool.ts showing the row count and the limit in AppModel.currentFeatures. The actual UI doesn't have a place to go yet.

Followup tasks:

- Real, pretty UI
- Counts per table
- Keep count(s) secret from users with limited access?
- Data size indicator?
- Banner when close to or above limit
- Measure row counts outside of sandbox to avoid spoofing with formula
- Handle changes to the limit when the plan is changed or extra rows are purchased

Test Plan: Tested UI manually, including with free team site, opening a fresh doc, opening an initialised doc, adding rows, undoing, and changes from another tab. Automated tests seem like they should wait for a proper UI.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3318
2022-03-14 21:49:32 +02:00
George Gevoian
ad1b4f3cff (core) Record new user sign-ups
Summary:
Adds Google Tag Manager snippet to all login pages, and a new user
preference, recordSignUpEvent, that's set to true on first sign-in. The
client now checks for this preference, and if true, dynamically loads
Google Tag Manager to record a sign-up event. Afterwards, it removes
the preference.

Test Plan: Tested manually.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3319
2022-03-12 14:34:46 -08:00
George Gevoian
f02174eb7e (core) Fix error when canceling import
Summary:
If cancel was clicked while a transform section was still being
generated in the Importer, an error was thrown. This refactors
the cancelImportFiles API action to take in the file upload id
in place of the entire DataSourceTransformed parameter, which
contains other values that are irrelevant to canceling. One of those
values, the transform section id, was causing the error to be thrown
since it was momentarily null.

Test Plan: Tested manually.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3317
2022-03-10 16:24:49 -08:00
Alex Hall
321019217d (core) Lossless imports
Summary:
- Removed string parsing and some type guessing code from parse_data.py. That logic is now implicitly done by ValueGuesser by leaving the initial column type as Any. parse_data.py mostly comes into play when importing files (e.g. Excel) containing values that already have types, i.e. numbers and dates.
- 0s and 1s are treated as numbers instead of booleans to keep imports lossless.
- Removed dateguess.py and test_dateguess.py.
- Changed what `guessDateFormat` does when multiple date formats work equally well for the given data, in order to be consistent with the old dateguess.py.
- Columns containing numbers are now always imported as Numeric, never Int.
- Removed `NullIfEmptyParser` because it was interfering with the new system. Its purpose was to avoid pointlessly changing a column from Any to Text when no actual data was inserted. A different solution to that problem was already added to `_ensure_column_accepts_data` in the data engine in a recent related diff.

Test Plan:
- Added 2 `nbrowser/Importer2` tests.
- Updated various existing tests.
- Extended testing of `guessDateFormat`. Added `guessDateFormats` to show how ambiguous dates are handled internally.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3302
2022-03-08 12:14:39 +02:00
George Gevoian
9522438967 (core) Add Grist forgot password page
Summary:
The page isn't yet linked to from anywhere in the UI, but
will be soon, once the new login page is ready. The page
can still be accessed at login-[s].getgrist.com/forgot-password,
and the flow is similar to the one used by Cognito's hosted UI.

Also refactors much of the existing login app code into smaller
files with less duplication, tweaks password validation to be closer
to Cognito's requirements, and polishes various parts of the UI,
like the verified page CSS, and the form inputs.

Test Plan: Browser, server and project tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3296
2022-03-07 09:11:28 -08:00
Paul Fitzpatrick
2563fb745a (core) make Grist easier to run with a single server
Summary:
This makes many small changes so that Grist is less fussy to run as a single instance behind a reverse proxy. Some users had difficulty with the self-connections Grist would make, due to internal network setup, and since these are unnecessary in any case in this scenario, they are now optimized away. Likewise some users had difficulties related to doc worker urls, which are now also optimized away. With these changes, users should be able to get a lot further on first try, at least far enough to open and edit documents.

The `GRIST_SINGLE_ORG` setting was proving a bit confusing, since it appeared to only work when set to `docs`. This diff
adds a check for whether the specified org exists, and if not, it creates it. This still depends on having a user email to make as the owner of the team, so there could be remaining difficulties there.

Test Plan: tested manually with nginx

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3299
2022-03-05 13:30:45 -05:00
Alex Hall
5213972d24 (core) Guess numeric formatting options
Summary:
Change NumberParse.parse to return not just the parsed number but also information it gathered along the way about how the input string was formatted.

Use this in the new NumberParse.guessOptions to guess the actual widget options based on an array of strings.

Use NumberParse.guessOptions in TypeConversion (for when a user explicitly chooses to change type) and in ValueGuesser (for guesses about strings entered into empty columns).

Test Plan: Adds unit tests for NumberParse and ValueGuesser and updates the TypeChange2 nbrowser test.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3294
2022-03-03 21:32:03 +02:00
Alex Hall
599545fb11 (core) Fuller guessing of type and options when adding first data to blank columns
Summary:
Adds `common/ValueGuesser.ts` with logic for guessing column type and widget options (only for dates/datetimes) from an array of strings, and converting the strings to the guessed type in a lossless manner, so that converting back to Text gives the original values.

Changes `_ensure_column_accepts_data` in Python to call an exported JS method using the new logic where possible.

Test Plan: Added `test/common/ValueGuesser.ts` to unit test the core guessing logic and a DocApi end-to-end test for what happens to new columns.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3290
2022-03-01 22:00:45 +02:00
George Gevoian
fa68b790bb (core) Remove code for unused welcome flows
Summary: Removes code that was marked for removal.

Test Plan: Existing tests still pass.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3289
2022-02-28 13:21:28 -08:00
George Gevoian
95592b81bd (core) Skip /welcome/user page for new users
Summary:
Since the new Grist sign-up page has a required field for
name, we can now skip the welcome page asking for the
same thing. Code and tests that can be removed later are
marked with TODOs.

Test Plan: Browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3266
2022-02-22 08:38:22 -08:00
Alex Hall
437d30bd9f (core) Log number of rows in user tables in data engine
Summary:
Adds a method Table._num_rows using an empty lookup map column.

Adds a method Engine.count_rows which adds them all up.

Returns the count after applying user actions to be logged by ActiveDoc.

Test Plan: Added a unit test in Python. Tested log message manually.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3275
2022-02-22 00:59:56 +02:00
Alex Hall
5b352211c4 (core) Guess date format during type conversion
Summary:
- Adds a dependency moment-guess (https://github.com/apoorv-mishra/moment-guess) to guess date formats from strings. However the npm package is missing source maps which leads to an ugly warning, so currently using a fork until https://github.com/apoorv-mishra/moment-guess/pull/22 is resolved.
- Adds guessDateFormat using moment-guess to determine the best candidate date format. The logic may be refined for e.g. lossless imports where the stakes are higher, but for now we're just trying to make type conversions smoother.
- Uses guessDateFormat to guess widget options when changing column type to date or datetime.
- Uses the date format of the original column when possible instead of guessing.
- Fixes a bug where choices were guessed based on the display column instead of the visible column, which made the guessed choices influenced by which values were referenced as well as completely broken when converting from reflist.
- @dsagal @georgegevoian This builds on https://phab.getgrist.com/D3265, currently unmerged. That diff was created first to alert to the change. Without it there would still be similar test failures/changes here as the date format would often be concretely guessed and saved as YYYY-MM-DD instead of being left as the default `undefined` which is shows as YYYY-MM-DD in the dropdown.

Test Plan: Added a unit test to `parseDate.ts`. Updated several browser tests which show the guessing in action during type conversion quite nicely.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal, georgegevoian

Differential Revision: https://phab.getgrist.com/D3264
2022-02-21 22:39:47 +02:00
Paul Fitzpatrick
be8c053922 (core) updates from grist-core 2022-02-21 13:44:13 -05:00
Edward Betts
d6e0e1fee3 Correct spelling mistakes 2022-02-19 09:46:49 +00:00
Alex Hall
8b5e837d9c (core) Use YYYY-MM-DD as default date format for parsing
Summary: By default, new date columns show "YYYY-MM-DD" as the date format in the UI, but under the hood it's stored as null/undefined, which affects parsing during pasting and type conversion. This changes the underlying parsing to always default to YYYY-MM-DD, as if that format was explicitly selected, making things more consistent.

Test Plan: Updated some tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3265
2022-02-18 12:16:42 +02:00
George Gevoian
e264094412 (core) Add account page option to allow Google login
Summary:
Enabled by default, the new checkbox is only visible to
users logged in with email/password, and controls whether it is possible
to log in to the same account via a Google account
(with matching email). When disabled, CognitoClient will refuse logins
from Google if a Grist account with the same email exists.

Test Plan:
Server and browser tests for setting flag. Manual tests to verify
Cognito doesn't allow signing in with Google when flag is disabled.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3257
2022-02-14 16:56:23 -08:00
George Gevoian
99f3422217 (core) Add new Grist sign-up page
Summary:
Available at login.getgrist.com/signup, the new sign-up page
includes similar options available on the hosted Cognito sign-up
page, such as support for registering with Google. All previous
redirects to Cognito for sign-up should now redirect to the new
Grist sign-up page.

Login is still handled with the hosted Cognito login page, and there
is a link to go there from the new sign-up page.

Test Plan: Browser, project and server tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3249
2022-02-14 10:32:47 -08:00
Alex Hall
f877f3859d (core) Use visible column formatting when converting RefList to Text
Summary: Tweaked ReferenceListFormatter and ValueConverter to sensibly convert to Text. Fixes an embarrassing bug exposed during checkin 😱

Test Plan: Add test for converting from RefList displaying formatted dates to Text

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3254
2022-02-12 09:45:57 +02:00
Alex Hall
0de0cb0f4a (core) Add PUT /records DocApi endpoint to AddOrUpdate records
Summary:
As designed in https://grist.quip.com/fZSrAnJKgO5j/Add-or-Update-Records-API

Current `POST /records` adds records, and `PATCH /records` updates them by row ID. This adds `PUT /records` to 'upsert' records, applying the AddOrUpdate user action. PUT was chosen because it's idempotent. Using a separate method (instead of inferring based on the request body) also cleanly separates validation, documentation, etc.

The name `require` for the new property was suggested by Paul because `where` isn't very clear when adding records.

Test Plan: New DocApi tests

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3251
2022-02-12 09:44:34 +02:00