Commit Graph

1140 Commits

Author SHA1 Message Date
George Gevoian
8f5ce9d610 (core) Wait for transform column before finalizing
Summary:
This adds a new private field to ColumnTransform that settles when
the transform column is done preparing. Finalize then waits for this
promise before proceeding.

A bug was occurring when the prepare promise was still pending,
but finalize was triggered indirectly, causing finalize to throw an error
due to the transform column not yet existing.

Test Plan: Tested manually.

Reviewers: jarek, dsagal

Reviewed By: jarek, dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3288
2022-03-01 13:59:14 -08:00
Alex Hall
599545fb11 (core) Fuller guessing of type and options when adding first data to blank columns
Summary:
Adds `common/ValueGuesser.ts` with logic for guessing column type and widget options (only for dates/datetimes) from an array of strings, and converting the strings to the guessed type in a lossless manner, so that converting back to Text gives the original values.

Changes `_ensure_column_accepts_data` in Python to call an exported JS method using the new logic where possible.

Test Plan: Added `test/common/ValueGuesser.ts` to unit test the core guessing logic and a DocApi end-to-end test for what happens to new columns.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3290
2022-03-01 22:00:45 +02:00
Paul Fitzpatrick
99ee495b4b v0.7.5 2022-02-28 16:52:44 -05:00
Jarosław Sadziński
ae6c857ac5 (core) Frozen columns reordering
Summary: Bug fix. Moving columns over the frozen set didn't work properly.

Test Plan: new tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3297
2022-02-28 22:52:37 +01:00
George Gevoian
fa68b790bb (core) Remove code for unused welcome flows
Summary: Removes code that was marked for removal.

Test Plan: Existing tests still pass.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3289
2022-02-28 13:21:28 -08:00
George Gevoian
83ba2957bf (core) Update failing HomeIntro core test
Summary:
Core doesn't redirect to Cognito or our own sign-up page
when clicking 'sign up' on the welcome screen. Instead, it
redirects to the test login page.

Test Plan: N/A (fixing test)

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3298
2022-02-28 12:23:28 -08:00
Paul Fitzpatrick
c91593e312 (core) make cognito logins for tests work on slightly smaller screens
Summary:
A selfish diff to tweak deployment tests so they pass on a
slightly smaller laptop screen that for whatever reason trips
a css breakpoint on the cognito login page.

Test Plan: manual

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3292
2022-02-25 15:04:45 -05:00
Cyprien P
c2070877b6 (core) Fix out of sync GROUP DATA and X Axis options when table change
Summary:
Fix this issue:
  When using “Change Widget” for a chart of a summary table, if I change group-by columns, the X-Axis dropdown becomes empty, and lists values from previous summary (not the actual summary table that’s now shown in the chart). Need to close and reopen creator panel to fix it.

Test Plan: Include nbrowser test

Reviewers: georgegevoian

Differential Revision: https://phab.getgrist.com/D3284
2022-02-25 14:48:27 +01:00
Alex Hall
3445ecb64c (core) Search backwards with Shift+Enter
Summary: As simple as it sounds. Check ev.shiftKey in Enter key handler in search.ts.

Test Plan: Extended test/nbrowser/Search.ts

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3293
2022-02-25 14:07:15 +02:00
George Gevoian
a294eefdff (core) Update HomeIntro.ts to check for either login page
Summary:
The "Sign In" button can either redirect you to the Cognito login
page or the Grist sign-up page, depending on whether you've logged
in before. Since the HomeIntro tests can either be run as part of a full
deployment test suite, or individually, the test could redirect you to
both types of login pages.

Test Plan: N/A (fixing test)

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3291
2022-02-24 15:28:16 -08:00
George Gevoian
ff4e5d2769 (core) Send emails when 2FA settings are updated
Summary: When user 2FA status is changed, we now send out emails via SendGrid.

Test Plan: Server tests.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3280
2022-02-24 12:36:50 -08:00
George Gevoian
8f1889d2d7 (core) Fix staging test that would timeout
Summary:
Removes code from simulateLogin that's no longer needed, and increases
the timeout on the util function that checks if we're on the Grist sign-up
page.

Test Plan: N/A (fixing test)

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3287
2022-02-23 19:03:41 -08:00
George Gevoian
ac910389bc (core) Make login URL regexes more specific
Summary:
This should resolve some staging test failures. The previous regexes
weren't quite specific enough, so random parts of a URL that had
traces of the real login URLs were causing isOnLoginPage() and
isOnGristLoginPage() to return true instead of false.

Test Plan: N/A (fixing tests)

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3283
2022-02-23 08:51:59 -08:00
Paul Fitzpatrick
accd640078 (core) add a user.SessionID value for trigger formulas and granular access rules
Summary:
This makes a `user.SessionID` value available in information about the user, for use with trigger formulas and granular access rules. The ID should be constant within a browser session for anonymous user. For logged in users it simply reflects their user id.

This ID makes it possible to write access rules and trigger formulas that allow different anonymous users to create, view, and edit their own records in a document.

For example, you could have a brain-storming document for puns, and allow anyone to add to it (without logging in), letting people edit their own records, but not showing the records to others until they are approved by a moderator. Without something like this, we could only let anonymous people add one field of a record, and not have a secure way to let them edit that field or others in the same record.

Also adds a `user.IsLoggedIn` flag in passing.

Test Plan: Added a test, updated tests. The test added is a mini-moderation doc, don't use it for real because it allows users to edit their entries after a moderator has approved them.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3273
2022-02-22 12:50:43 -05:00
George Gevoian
95592b81bd (core) Skip /welcome/user page for new users
Summary:
Since the new Grist sign-up page has a required field for
name, we can now skip the welcome page asking for the
same thing. Code and tests that can be removed later are
marked with TODOs.

Test Plan: Browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3266
2022-02-22 08:38:22 -08:00
George Gevoian
36843e632b (core) Hide API key when it's not selected
Summary:
The API key is now hidden by default. Clicking the input
will select and reveal the key. As soon as the key loses
selection, it is hidden again.

Test Plan: Project test.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3270
2022-02-21 17:26:45 -08:00
Alex Hall
437d30bd9f (core) Log number of rows in user tables in data engine
Summary:
Adds a method Table._num_rows using an empty lookup map column.

Adds a method Engine.count_rows which adds them all up.

Returns the count after applying user actions to be logged by ActiveDoc.

Test Plan: Added a unit test in Python. Tested log message manually.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3275
2022-02-22 00:59:56 +02:00
Alex Hall
f1002c0e67 (core) Regularly log data size in DocStorage.applyStoredActions using sqlite dbstat
Summary:
- Small cleanup: Make DocStorage implement OnDemandStorage, and remove unused execWithBackup
- Upgrade to new versions (.3) of @gristlabs/sqlite3 and connect-sqlite3 to use dbstat
- Add _logDataSize method which queries dbstat, adding up pgsize for tables loaded into the data engine
- Only complete _logDataSize every 5 minutes using new field _lastLoggedDataSize

Test Plan: Tested manually

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3277
2022-02-22 00:59:04 +02:00
Alex Hall
4c935e7fb3 (core) Import Hashable from six.moves.collections_abc
Summary: Copy of https://github.com/gristlabs/grist-core/pull/136

Test Plan: this

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3278
2022-02-22 00:27:51 +02:00
Paul Fitzpatrick
21f2765a59 (core) updates from grist-core 2022-02-21 17:18:37 -05:00
Paul Fitzpatrick
88fe090032
Merge pull request #136 from tirkarthi/fix-abc
Import ABC through six.moves for Python 2/3 compatibility.
2022-02-21 17:18:05 -05:00
Alex Hall
5b352211c4 (core) Guess date format during type conversion
Summary:
- Adds a dependency moment-guess (https://github.com/apoorv-mishra/moment-guess) to guess date formats from strings. However the npm package is missing source maps which leads to an ugly warning, so currently using a fork until https://github.com/apoorv-mishra/moment-guess/pull/22 is resolved.
- Adds guessDateFormat using moment-guess to determine the best candidate date format. The logic may be refined for e.g. lossless imports where the stakes are higher, but for now we're just trying to make type conversions smoother.
- Uses guessDateFormat to guess widget options when changing column type to date or datetime.
- Uses the date format of the original column when possible instead of guessing.
- Fixes a bug where choices were guessed based on the display column instead of the visible column, which made the guessed choices influenced by which values were referenced as well as completely broken when converting from reflist.
- @dsagal @georgegevoian This builds on https://phab.getgrist.com/D3265, currently unmerged. That diff was created first to alert to the change. Without it there would still be similar test failures/changes here as the date format would often be concretely guessed and saved as YYYY-MM-DD instead of being left as the default `undefined` which is shows as YYYY-MM-DD in the dropdown.

Test Plan: Added a unit test to `parseDate.ts`. Updated several browser tests which show the guessing in action during type conversion quite nicely.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal, georgegevoian

Differential Revision: https://phab.getgrist.com/D3264
2022-02-21 22:39:47 +02:00
George Gevoian
481122b4d3 (core) Fix failing login tests
Summary:
The rollout of the new sign-up page on prod caused a few tests
that previously expected to be on the Cognito sign-up page to fail.

Test Plan: N/A (fixing failing tests).

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3279
2022-02-21 11:50:30 -08:00
Paul Fitzpatrick
be8c053922 (core) updates from grist-core 2022-02-21 13:44:13 -05:00
Paul Fitzpatrick
e73e74b8ad
Merge pull request #141 from N6UDP/patch-1
Fix typo in email fallback for SAML
2022-02-20 15:11:00 -05:00
Lee Burton
9c47b9cdee
Fix typo in email fallback for SAML
It looks like nameId doesn't exist as a property but name_id does (as is used elsewhere in the function)
2022-02-20 02:39:30 -08:00
Paul Fitzpatrick
6d941fb5a3
Merge pull request #137 from EdwardBetts/spelling
Correct spelling mistakes
2022-02-19 21:19:31 -05:00
Edward Betts
d6e0e1fee3 Correct spelling mistakes 2022-02-19 09:46:49 +00:00
Karthikeyan Singaravelan
08cbdf29f5 Import ABC through six.moves for Python 2/3 compatibility. 2022-02-19 05:31:05 +00:00
Paul Fitzpatrick
bf7c46c3cd
Merge pull request #132 from mscherer/patch-1
Fix more links
2022-02-18 09:24:45 -05:00
mscherer
988e3ab7f0 Fix links
Alos unify them a bit while on it by removing https:// prefix
2022-02-18 11:36:57 +01:00
Alex Hall
8b5e837d9c (core) Use YYYY-MM-DD as default date format for parsing
Summary: By default, new date columns show "YYYY-MM-DD" as the date format in the UI, but under the hood it's stored as null/undefined, which affects parsing during pasting and type conversion. This changes the underlying parsing to always default to YYYY-MM-DD, as if that format was explicitly selected, making things more consistent.

Test Plan: Updated some tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3265
2022-02-18 12:16:42 +02:00
Cyprien P
afa90cc365 (core) Show default context menu on link
Summary:
also:
  - closes opened menu if any when click on a custom widget
  - closes opened menu if any when F2

Test Plan: Include test case

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3269
2022-02-18 10:09:36 +01:00
Paul Fitzpatrick
f224afcc62 (core) updates from grist-core 2022-02-17 16:53:25 -05:00
Paul Fitzpatrick
6c01e2c7b6
Merge pull request #131 from matthiaskoenig/patch-1
Fixing broken link
2022-02-17 16:21:32 -05:00
Matthias König
005564f9c2
Fixing broken link
link was relative link on page which did not resolve
2022-02-17 22:01:08 +01:00
Dmitry S
9855a145d1 (core) Remove UI-imposed restriction against combining public edit access and access rules
Test Plan: TBD

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3272
2022-02-17 09:57:23 -05:00
Jarosław Sadziński
8d34b70899 (core) Exposing WidgetApi methods in a module scope
Summary:
Exposing WidgetApi methods (getOption, setOption, ...) in a module
scope to simplify custom widget code.

Test Plan: manual tests for now, as plugin is not yet on production.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3268
2022-02-16 16:49:29 +01:00
Dmitry S
3136077636 (core) Update documentation of CURRENT_CONVERSION function
Summary: As requested by Anais in https://github.com/dsagal/grist-help/pull/132

Test Plan: Only a change to doc-comment (from which help docs are built).

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3267
2022-02-16 08:48:12 -05:00
Alex Hall
c0ca936c1e (core) Backend restrictions for raw data widgets
Summary:
Prevent most updates or removals of raw view sections and their fields in `useractions.py`. Only a fiew columns are allowed to be updated.

Removed the unused method `_UpdateViews` while I was at it.

Test Plan:
Added a Python test.

Tested manually that I can still make all expected changes, i.e. those allowed by the UI, e.g. reordering columns.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3263
2022-02-15 22:04:32 +02:00
Cyprien P
2f6eafff35 (core) Adds setSelectedRows to the grist api for custom view
Summary:
This is needed to let custom widget driver filtering of other widget in the same page.

Descripion here:
 - https://grist.quip.com/ctytAQJoFMsM/Hopefully-Small-Projects#temp:C:NNCfe2030b27647439886ca83595

Test Plan: New api tested in a new nbrowser test

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3253
2022-02-15 15:24:15 +01:00
George Gevoian
e264094412 (core) Add account page option to allow Google login
Summary:
Enabled by default, the new checkbox is only visible to
users logged in with email/password, and controls whether it is possible
to log in to the same account via a Google account
(with matching email). When disabled, CognitoClient will refuse logins
from Google if a Grist account with the same email exists.

Test Plan:
Server and browser tests for setting flag. Manual tests to verify
Cognito doesn't allow signing in with Google when flag is disabled.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3257
2022-02-14 16:56:23 -08:00
George Gevoian
99f3422217 (core) Add new Grist sign-up page
Summary:
Available at login.getgrist.com/signup, the new sign-up page
includes similar options available on the hosted Cognito sign-up
page, such as support for registering with Google. All previous
redirects to Cognito for sign-up should now redirect to the new
Grist sign-up page.

Login is still handled with the hosted Cognito login page, and there
is a link to go there from the new sign-up page.

Test Plan: Browser, project and server tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3249
2022-02-14 10:32:47 -08:00
Paul Fitzpatrick
d51180d349 (core) updates from grist-core 2022-02-14 08:21:42 -05:00
George Gevoian
6abe7d5827 (core) Use original column headers during imports
Summary:
When possible, the original column headers from imported
files will now be used as the labels for Grist columns. This includes
values that were previously invalid Grist column identifiers, such
as those containing Unicode.

Test Plan: Updated server and browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3261
2022-02-13 16:50:19 -08:00
Dmitry
25b7e6c245
Tweaks to documentation comments for user-facing Grist functions (#126)
- Update internal links in function documentation
- Remove emphasis in code blocks
- Remove trailing whitespace
2022-02-13 00:45:24 -05:00
Natalie Misasi
2611d05c39
Updates to Grist Functions (#125)
Clarify documentation for QUOTIENT, SUMPRODUCT, UUID, REPLACE, lookupOne, lookupRecords, and $group.
2022-02-13 00:38:24 -05:00
Alex Hall
f877f3859d (core) Use visible column formatting when converting RefList to Text
Summary: Tweaked ReferenceListFormatter and ValueConverter to sensibly convert to Text. Fixes an embarrassing bug exposed during checkin 😱

Test Plan: Add test for converting from RefList displaying formatted dates to Text

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3254
2022-02-12 09:45:57 +02:00
Alex Hall
0de0cb0f4a (core) Add PUT /records DocApi endpoint to AddOrUpdate records
Summary:
As designed in https://grist.quip.com/fZSrAnJKgO5j/Add-or-Update-Records-API

Current `POST /records` adds records, and `PATCH /records` updates them by row ID. This adds `PUT /records` to 'upsert' records, applying the AddOrUpdate user action. PUT was chosen because it's idempotent. Using a separate method (instead of inferring based on the request body) also cleanly separates validation, documentation, etc.

The name `require` for the new property was suggested by Paul because `where` isn't very clear when adding records.

Test Plan: New DocApi tests

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3251
2022-02-12 09:44:34 +02:00
Jarosław Sadziński
66eb0b91b8 (core) API fix for a bug that treated 0 as null
Summary: Bug with 'records' endpoint that was treating 0 as null.

Test Plan: Modified tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3262
2022-02-12 01:39:56 +01:00