Summary:
Column filter menu use to mess up the ordering of the items for
numeric and dates values, and also for ref/reflist columns when the
visible column is a numeric a date column.
Solution was to:
- use the actual value of the visible column for comparison.
- use native comparison.
- tweak the native comparison to make blanks appears before valid value. Indeed, it came up several time that it's convenient to have invalid values show up first in the filter panel, it makes for a convenient way to detect them.
Test Plan: Adds new nbrowser test
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D3441
Summary:
Seem in a customer doc where Y-scale was wrong for unclear reasons. The cause
is that null-valued X labels cause the corresponding bar (or point) to be
omitted, but still affect the Y-scale. In this diff, such labels are replaced
with "-", so as to show up normally.
Trivial example that reproduces the problem here:
https://public.getgrist.com/iLPpx9C5i8nk/Null-in-X-Axis
Test Plan: Tested manually.
Reviewers: cyprien
Reviewed By: cyprien
Subscribers: cyprien
Differential Revision: https://phab.getgrist.com/D3442
Summary:
Instead of showing a blank dialog for users whose access
is limited (e.g. public members), we now show the user's
role and a mention of whether their access is public.
Test Plan: Browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3431
Summary:
This allows limiting the memory available to documents in the sandbox when gvisor is used. If memory limit is exceeded, we offer to open doc in recovery mode. Recovery mode is tweaked to open docs with tables in "ondemand" mode, which will generally take less memory and allow for deleting rows.
The limit is on the size of the virtual address space available to the sandbox (`RLIMIT_AS`), which in practice appears to function as one would want, and is the only practical option. There is a documented `RLIMIT_RSS` limit to `specifies the limit (in bytes) of the process's resident set (the number of virtual pages resident in RAM)` but this is no longer enforced by the kernel (neither the host nor gvisor).
When the sandbox runs out of memory, there are many ways it can fail. This diff catches all the ones I saw, but there could be more.
Test Plan: added tests
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3398
Summary:
This makes it possible to configure a SendGrid-based Notifier
instance via a JSON configuration file.
Test Plan: Tested manually.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D3432
Summary:
The summary includes a count of documents that are approaching
limits, in grace period, or delete-only. The endpoint is only accessible
to site owners, and is currently unused. A follow-up diff will add usage
banners to the site home page, which will use the response from the
endpoint to communicate usage information to owners.
Test Plan: Browser and server tests.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D3420
Summary:
Grid UI improvements.
- Selecting a column with a custom background, didn't produce a visual difference.
- Hiding num-row right border when it is not needed (it should be used only for frozen columns)
Test Plan: manual tests
Reviewers: cyprien
Reviewed By: cyprien
Differential Revision: https://phab.getgrist.com/D3425
Summary:
Adds a new environment variable that allows for custom
CSS to be included in all core static pages.
Test Plan: Tested manually in grist-core.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3419
Summary:
Relax the restriction in `selectBy.isValidLink` so that summary tables can be linked by a column like other tables, except the `group` column. See the discussion on https://grist.slack.com/archives/C0234CPPXPA/p1651773623256959 (the replies are on the following message) for more info on this decision.
Tweaked `LinkingState.ts` since linking with summary tables can now involve a column.
Test Plan: Added a new nbrowser test and fixture checking the options to select by given a summary table with a few ref/reflist columns. Manually tested the behaviour of each option.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D3416
Summary:
Use openpyxl instead of messytables (which used xlrd internally) in import_xls.py.
Skip empty rows since excel files can easily contain huge numbers of them.
Drop support for xls files (which openpyxl doesn't support) in favour of the newer xlsx format.
Fix some details relating to python virtualenvs and dependencies, as Jenkins was failing to find new Python dependencies.
Test Plan: Mostly relying on existing tests. Updated various tests which referred to xls files instead of xlsx. Added a Python test for skipping empty rows.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3406
Summary:
Diff fixes couple edge cases:
- When changing chart's groupby columns, the data-engine changes the view fields, which is not okay with charts, hence makes sure view fields stay the same as much as possible using `this.setSectionViewFieldsFromArray()`
- Also there's a logic in `this.setSectionViewFieldsFromArray()` that handle what to do when some columns goes missing during updates. Diff fixes this logic two. This is really corner case.
Test Plan: Both cases are not added to ChartAggregate.ts
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3405
Summary:
Adds attachment and data size to the usage section of
the raw data page. Also makes in-document usage banners
update as user actions are applied, causing them to be
hidden/shown or updated based on the current state of
the document.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3395
Summary:
- Better focus on the widget title
- Adding columns only to the current view section
- New popup with options when user wants to delete a page
- New dialog to enter table name
- New table as a widget doesn't create a separate page
- Removing a table doesn't remove the primary view
Test Plan: Updated and new tests
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3410
Summary:
Skipping columns during incremental imports wasn't working for certain
column types, such as numeric columns. The column's default value was
being used instead (e.g. 0), overwriting values in the destination
table.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3402
Summary:
A new way for renaming tables.
- There is a new popup to rename section (where you can also rename the table)
- Renaming/Deleting page doesn't modify/delete the table.
- Renaming table can rename a page if the names match (and the page contains a section with that table).
- User can rename table in Raw Data UI in two ways - either on the listing or by using the section name popup
- As before, there is no way to change tableId - it is derived from a table name.
- When the section name is empty the table name is shown instead.
- White space for section name is allowed (to discuss) - so the user can just paste ' '.
- Empty name for a page is not allowed (but white space is).
- Some bugs related to deleting tables with attached summary tables (and with undoing this operation) were fixed (but not all of them yet).
Test Plan: Updated tests.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D3360
Summary:
Changing keyboard shortcut for Duplicate row from
Ctrl+Shift+D to Mod+Shift+D
Test Plan: existing test
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3401
Summary:
- Previously showed "UnboundLocalError". Now will show:
Import failed: Failed to parse Excel file.
Error: No tables found (1 empty tables skipped)
- Also fix logging for import code
Test Plan: Added a test case
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3396
Summary:
Fixing bug when comparing documents. User wasn't able
to open cell/column context menu due to javascript error.
Test Plan: Updated tests
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3397
Summary:
This also enables the new Usage section for all sites. Currently,
it shows metrics for document row count, but only if the user
has full document read access. Otherwise, a message about
insufficient access is shown.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3377
Summary:
When grid view was not fully loaded the initial blank row
was marked as invalid by conditional styles. This removes the styling
when row is not fully loaded.
Test Plan: Tested manually
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3392
Summary:
New command "Duplicate rows" is available in the Row/Card Context Menu and as a keyboard shortcut Ctrl+Alt+C.
- All selected rows are duplicated (even if only a single column is selected)
- Rows are inserted immediately after the last selected row (using manualSort value).
- Formulas and CENSORED fields are not copied.
Implemented on the UI level (no new action).
Test Plan: new test
Reviewers: cyprien
Reviewed By: cyprien
Differential Revision: https://phab.getgrist.com/D3371
Summary: Adds a command and `BaseView` method `filterByThisCellValue`.
Test Plan: Added two tests to `nbrowser/CellContextMenu.ts`
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3383
Summary:
Search now works on Raw Data Page.
- Search bar option 'Search on all pages' will change to 'Search on all tables' when on the Raw data page, and will allow searching through all tables.
- Little CSS adjustment for an overlay on Raw page (removes z-index as it is not needed, and conflicts with searchbar).
- Search bar option ('search on all') gets white background, little padding, and is moved 2 pixels up, this is needed for Raw page.
Test Plan: new and updated tests
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3376
Summary: Also cleans up login URLs by excluding params and hashes.
Test Plan: Client and server tests.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3378
Summary:
For users who cannot otherwise change access to a resource, let
them remove themselves. Implemented via the standard endpoints
as a special exception that will process a request from a user
that would otherwise be denied, if the only contents of that
request are a removal of themselves.
Users who can change access are still not permitted to change their
own permissions or to remove themselves, as a precaution against
orphaning resources.
Test Plan: extended and updated tests
Reviewers: cyprien
Reviewed By: cyprien
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3367
Summary:
- Symptoms where that Split Series could end up being turned off for
no good reason. Also both x axis and split series could be mixed
up.
- Problems was caused by call to `setGroupByColumns` which modifies
the sections viewFields. Diff fixes it by adjustin slightly the
ordering of function call in `_setXAxis()`.
- Problem of mixing up x axis and split series was fixed by being
careful on the order of columns passed to the `setGroupByColumns`
which then determine the ordering of the view fields.
Test Plan: Includes new test cases
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D3365
Summary:
New onNewRecord event in custom widgets that is triggered
when the user selects a blank row.
Together with this diff there is PR for grist-help https://github.com/gristlabs/grist-help/pull/150
Test Plan: added test case
Reviewers: cyprien
Reviewed By: cyprien
Differential Revision: https://phab.getgrist.com/D3375
Summary:
- Include docId when available for client-side error reporting
- Distinguish sandbox crashes from forced exits
Test Plan: Tested manually
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3373
Summary:
This adds an optional `isConsultant` flag to `users.options`, and an endpoint that allows the support user to turn it on or off. Users marked as consultants are not counted as billable members. Follows the example of existing `allowGoogleLogin` option.
Billable members are counted when members are added or removed from a site. Changing the `isConsultant` flag has no immediate or retroactive effect on billing. The number of users in stripe is now set unconditionally, rather than only when it has changed.
Notifications to billing managers are not aware of this billing nuance, but continue to report user counts that include consultants. The notifications link users to the billing page.
Test Plan: extended test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: anaisconce, jarek
Differential Revision: https://phab.getgrist.com/D3362
Summary:
When linking table to a summary, the linking ended up broken after updating the summary group by columns.
This diff fixes that issue.
There were two issues:
1) some subscriptions were missing due to some .peek() calls instead of directly calling the observable.
2) the LinkingState instance was not being disposed.
3) the filterColValues was not updating after source data table has been loaded
Test Plan: Include new test file.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D3358
Summary:
Redesigning color picker:
- Single color palette (no light/dark switch)
- Ability to remove color (new empty button)
New font options in the color picker.
Font options are available on:
- Default cell style
- Conditional rules styles
- Choice/ChoiceList editor and token field
- Filters for Choice/ChoiceList columns
Design document:
https://www.figma.com/file/bRTsb47VIOVBfJPj0qF3C9/Grist-Updates?node-id=415%3A8135
Test Plan: new and updated tests
Reviewers: georgegevoian, alexmojaki
Reviewed By: georgegevoian, alexmojaki
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3335
Summary:
- adds the `aggregate values` option to chart config
- aggregation is performed by turning table into summary table
- change columns options of xaxis and split series selectors to be the source table columns when `aggregate values` is on
- change xAxis and split series computed to hold colId instead of column id
- change GristDoc saveViewSection routine to preserve old sections viewFields and options
- Rename `Group data` into `split series`
quip doc: https://grist.quip.com/tAsCAuv8RiMa/Charts-data-aggregation#temp:C:QcK0ce13e1e8ae64048988f44f9c
Test Plan: Adds ChartAggregate.ts nbrowser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D3336
Summary:
Currently, usage is only shown for free team sites, and only
for total number of rows used in a document. Future diffs will
include other usage metrics and browser tests.
Test Plan: Planned for future diffs; UI is still under development.
Reviewers: jarek
Reviewed By: jarek
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3343
Summary:
Adds a `data-grist-col-ref` attribute to the copied HTML, then uses that when pasting to look up the source column and retrieve info about it. Copies the info into the target column if:
- The document is the same (the docId hash matches)
- The source column still exists and has the same type as when copied
- The source type isn't Text, because in that case it's nice if type guessing still happens
- The target column is empty, meaning it has type Any (we check earlier that it's not a formula column)
The info copied is the type, widgetOptions, and reference column settings (visible and display columns) but not conditional formatting.
The changes are mostly in a function `parsePasteForView` which is based on `BaseView._parsePasteForView` but ported to TypeScript in a new file `BaseView2.ts`.
Added a useraction `MaybeCopyDisplayFormula` exposing an existing Python function `maybe_copy_display_formula` because the target column needs a slightly different display formula.
Test Plan: Added a new nbrowser test file and fixture doc.
Reviewers: cyprien
Reviewed By: cyprien
Subscribers: jarek, dsagal
Differential Revision: https://phab.getgrist.com/D3344
Summary:
Adds a new Grist login page to the login app, and replaces the
server-side Cognito Google Sign-In flow with Google's own OAuth flow.
Test Plan: Browser and server tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D3332
Summary: Fixing bug introduced in https://phab.getgrist.com/D3338: column config now shows "DATA FROM TABLE" under the Attachments column type as if it were a normal ref/reflist column.
Test Plan: manual
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3350
Summary:
- When opening HelpScout beacon to an article ("answers"), avoid a 'navigate'
call to let the beacon show the previously open article.
- Work around a bug with reloading a page with a beacon article open: HelpScout
renders the last state without triggering usual events.
- Report errors to server when beacon fails to load.
- reportWarning() method now reports the message to the server.
Test Plan: Added a test case
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3345
Summary:
Widgets that were using old base Widget class didn't not
create conditional style rules ui.
Additional fixed a little bug - when adding conditional rule
the formula field was marked as having error for a split second.
Test Plan: new test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3346
Summary:
Builds upon https://phab.getgrist.com/D3328
- Add HomeDB column `Document.gracePeriodStart`
- When the row count moves above the limit, set it to the current date. When it moves below, set it to null.
- Add DataLimitStatus type indicating if the document is approaching the limit, is in a grace period, or is in delete only mode if the grace period started at least 14 days ago. Compute it in ActiveDoc and send it to client when opening.
- Only allow certain user actions when in delete-only mode.
Follow-up tasks related to this diff:
- When DataLimitStatus in the client is non-empty, show a banner to the appropriate users.
- Only send DataLimitStatus to users with the appropriate access. There's no risk landing this now since real users will only see null until free team sites are released.
- Update DataLimitStatus immediately in the client when it changes, e.g. when user actions are applied or the product is changed. Right now it's only sent when the document loads.
- Update row limit, grace period start, and data limit status in ActiveDoc when the product changes, i.e. the user upgrades/downgrades.
- Account for data size when computing data limit status, not just row counts.
See also the tasks mentioned in https://phab.getgrist.com/D3331
Test Plan: Extended FreeTeam nbrowser test, testing the 4 statuses.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3331
Summary:
This adds support for gvisor sandboxing in core. When Grist is run outside of a container, regular gvisor can be used (if on linux), and will run in rootless mode. When Grist is run inside a container, docker's default policy is insufficient for running gvisor, so a fork of gvisor is used that has less defence-in-depth but can run without privileges.
Sandboxing is automatically turned on in the Grist core container. It is not turned on automatically when built from source, since it is operating-system dependent.
This diff may break a complex method of testing Grist with gvisor on macs that I may have been the only person using. If anyone complains I'll find time on a mac to fix it :)
This diff includes a small "easter egg" to force document loads, primarily intended for developer use.
Test Plan: existing tests pass; checked that core and saas docker builds function
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3333
Summary:
Clicking undo/redo after converting a table to a summary table navigated
to the raw data view.
Test Plan: new test
Reviewers: georgegevoian, alexmojaki
Reviewed By: georgegevoian, alexmojaki
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3337
Summary:
Adding conditional formatting rules feature.
Each column can have multiple styling rules which are applied in order
when evaluated to a truthy value.
- The creator panel has a new section: Cell Style
- New user action AddEmptyRule for adding an empty rule
- New columns in _grist_Table_columns and fields
A new color picker will be introduced in a follow-up diff (as it is also
used in choice/choice list/filters).
Design document:
https://grist.quip.com/FVzfAgoO5xOF/Conditional-Formatting-Implementation-Design
Test Plan: new tests
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3282
Summary:
This diff introduces cursor features for raw data views:
- Restoring cursor position when the browser window is reloaded
- Restoring the last edit position when the browser window is reloaded
Test Plan: Added tests
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D3314
Summary:
- Unify where in the code tours get initiated.
- Avoid start a new tour while one is being started or is in progress.
- Ignore welcome tour when on a doc that has a doc tour.
- Fix tours when starting with a special page like Access Rules.
- Remove mention of the no-longer-present "Give Feedback" button in the last
message of the welcome tour.
Test Plan:
Add a browser test case that docTour preempts the welcome tour and shows no errors
(this test case fails in multiple ways without the changes).
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3330
Summary:
Adds nbrowser test
- Also makes sort spec taken into account by Group Data options
- This is a continuation of https://phab.getgrist.com/D3271
- We still need to decide whether to add stack chart to area chart type
Test Plan: TBD
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3274
Summary: After dragging rows up, selection was set incorrectly.
Test Plan: Expanded a browser test for dragging rows to check selection, which fails without this fix
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3324
Summary:
Moves some auth-related UI components, like MFAConfig, out
of core, and adds a new ChangePasswordDialog component for
allowing direct password changes, replacing the old reset password
link to hosted Cognito.
Updates all MFA endpoints to use SRP for authentication.
Also refactors MFAConfig into smaller files, and polishes up some parts
of the UI to be more consistent with the login pages.
Test Plan: New server and deployment tests. Updated existing tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3311
Summary: This makes an equivalent of the /records REST endpoint available within custom widgets. For simple operations, it is compatible with https://github.com/airtable/airtable.js/. About half of the diff is refactoring code from DocApi that implements /records using applyUserActions, to make that code available in the plugin api.
Test Plan: added tests
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D3320
Summary:
Add rowCount returned from sandbox when applying user actions to ActionGroup which is broadcast to clients.
Add rowCount to ActiveDoc and update it after applying user actions.
Add rowCount to OpenLocalDocResult using ActiveDoc value, to show when a client opens a doc before any user actions happen.
Add rowCount observable to DocPageModel which is set when the doc is opened and when action groups are received.
Add crude UI (commented out) in Tool.ts showing the row count and the limit in AppModel.currentFeatures. The actual UI doesn't have a place to go yet.
Followup tasks:
- Real, pretty UI
- Counts per table
- Keep count(s) secret from users with limited access?
- Data size indicator?
- Banner when close to or above limit
- Measure row counts outside of sandbox to avoid spoofing with formula
- Handle changes to the limit when the plan is changed or extra rows are purchased
Test Plan: Tested UI manually, including with free team site, opening a fresh doc, opening an initialised doc, adding rows, undoing, and changes from another tab. Automated tests seem like they should wait for a proper UI.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3318
Summary:
Adds Google Tag Manager snippet to all login pages, and a new user
preference, recordSignUpEvent, that's set to true on first sign-in. The
client now checks for this preference, and if true, dynamically loads
Google Tag Manager to record a sign-up event. Afterwards, it removes
the preference.
Test Plan: Tested manually.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3319
Summary:
Creating new UI for raw data views based on design.
- Renaming left for follow up diff
- Link in the menu is hidden for now
To access raw UI, use /p/data URL.
Test Plan: new tests
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3306
Summary:
If cancel was clicked while a transform section was still being
generated in the Importer, an error was thrown. This refactors
the cancelImportFiles API action to take in the file upload id
in place of the entire DataSourceTransformed parameter, which
contains other values that are irrelevant to canceling. One of those
values, the transform section id, was causing the error to be thrown
since it was momentarily null.
Test Plan: Tested manually.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D3317
Summary:
Custom widget into page is served from a homeUrl instead
of untrusted URL, which might be not used in grist-core.
Test Plan: manual test
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3307
Summary: Fix for error that happens when a search is resumed after one of the columns was hidden.
Test Plan: Added test that shows the error.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D3309
Summary:
- Removed string parsing and some type guessing code from parse_data.py. That logic is now implicitly done by ValueGuesser by leaving the initial column type as Any. parse_data.py mostly comes into play when importing files (e.g. Excel) containing values that already have types, i.e. numbers and dates.
- 0s and 1s are treated as numbers instead of booleans to keep imports lossless.
- Removed dateguess.py and test_dateguess.py.
- Changed what `guessDateFormat` does when multiple date formats work equally well for the given data, in order to be consistent with the old dateguess.py.
- Columns containing numbers are now always imported as Numeric, never Int.
- Removed `NullIfEmptyParser` because it was interfering with the new system. Its purpose was to avoid pointlessly changing a column from Any to Text when no actual data was inserted. A different solution to that problem was already added to `_ensure_column_accepts_data` in the data engine in a recent related diff.
Test Plan:
- Added 2 `nbrowser/Importer2` tests.
- Updated various existing tests.
- Extended testing of `guessDateFormat`. Added `guessDateFormats` to show how ambiguous dates are handled internally.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3302
Summary:
The page isn't yet linked to from anywhere in the UI, but
will be soon, once the new login page is ready. The page
can still be accessed at login-[s].getgrist.com/forgot-password,
and the flow is similar to the one used by Cognito's hosted UI.
Also refactors much of the existing login app code into smaller
files with less duplication, tweaks password validation to be closer
to Cognito's requirements, and polishes various parts of the UI,
like the verified page CSS, and the form inputs.
Test Plan: Browser, server and project tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D3296
Summary:
Until now, users could not pick the column that's currently serving as
the x axis.
To do that, chart needs to support having the x axis undefined.
This diff do just that:
- allow x axis to be undefined
- allow setting x axis from group data
Given that charts axis are stored as indexes of the section view
fields array, implementation required introduction of an extra chart
options: `isAxisUndefined`.
Test Plan: Updates existing test and adds new one.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D3304
Summary:
This makes many small changes so that Grist is less fussy to run as a single instance behind a reverse proxy. Some users had difficulty with the self-connections Grist would make, due to internal network setup, and since these are unnecessary in any case in this scenario, they are now optimized away. Likewise some users had difficulties related to doc worker urls, which are now also optimized away. With these changes, users should be able to get a lot further on first try, at least far enough to open and edit documents.
The `GRIST_SINGLE_ORG` setting was proving a bit confusing, since it appeared to only work when set to `docs`. This diff
adds a check for whether the specified org exists, and if not, it creates it. This still depends on having a user email to make as the owner of the team, so there could be remaining difficulties there.
Test Plan: tested manually with nginx
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D3299
Summary:
This error only occured for certain charts type. Some chart handled it correctly, others did not.
This brings a more general handling of this issue which should work for all charts.
Test Plan: New nbrowser test case.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D3303
Summary:
Change NumberParse.parse to return not just the parsed number but also information it gathered along the way about how the input string was formatted.
Use this in the new NumberParse.guessOptions to guess the actual widget options based on an array of strings.
Use NumberParse.guessOptions in TypeConversion (for when a user explicitly chooses to change type) and in ValueGuesser (for guesses about strings entered into empty columns).
Test Plan: Adds unit tests for NumberParse and ValueGuesser and updates the TypeChange2 nbrowser test.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3294
Summary:
This adds a new private field to ColumnTransform that settles when
the transform column is done preparing. Finalize then waits for this
promise before proceeding.
A bug was occurring when the prepare promise was still pending,
but finalize was triggered indirectly, causing finalize to throw an error
due to the transform column not yet existing.
Test Plan: Tested manually.
Reviewers: jarek, dsagal
Reviewed By: jarek, dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3288
Summary:
Adds `common/ValueGuesser.ts` with logic for guessing column type and widget options (only for dates/datetimes) from an array of strings, and converting the strings to the guessed type in a lossless manner, so that converting back to Text gives the original values.
Changes `_ensure_column_accepts_data` in Python to call an exported JS method using the new logic where possible.
Test Plan: Added `test/common/ValueGuesser.ts` to unit test the core guessing logic and a DocApi end-to-end test for what happens to new columns.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3290
Summary: Bug fix. Moving columns over the frozen set didn't work properly.
Test Plan: new tests
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3297
Summary: Removes code that was marked for removal.
Test Plan: Existing tests still pass.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3289
Summary:
Fix this issue:
When using “Change Widget” for a chart of a summary table, if I change group-by columns, the X-Axis dropdown becomes empty, and lists values from previous summary (not the actual summary table that’s now shown in the chart). Need to close and reopen creator panel to fix it.
Test Plan: Include nbrowser test
Reviewers: georgegevoian
Differential Revision: https://phab.getgrist.com/D3284
Summary: As simple as it sounds. Check ev.shiftKey in Enter key handler in search.ts.
Test Plan: Extended test/nbrowser/Search.ts
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D3293
Summary:
This makes a `user.SessionID` value available in information about the user, for use with trigger formulas and granular access rules. The ID should be constant within a browser session for anonymous user. For logged in users it simply reflects their user id.
This ID makes it possible to write access rules and trigger formulas that allow different anonymous users to create, view, and edit their own records in a document.
For example, you could have a brain-storming document for puns, and allow anyone to add to it (without logging in), letting people edit their own records, but not showing the records to others until they are approved by a moderator. Without something like this, we could only let anonymous people add one field of a record, and not have a secure way to let them edit that field or others in the same record.
Also adds a `user.IsLoggedIn` flag in passing.
Test Plan: Added a test, updated tests. The test added is a mini-moderation doc, don't use it for real because it allows users to edit their entries after a moderator has approved them.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3273
Summary:
Since the new Grist sign-up page has a required field for
name, we can now skip the welcome page asking for the
same thing. Code and tests that can be removed later are
marked with TODOs.
Test Plan: Browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3266
Summary:
The API key is now hidden by default. Clicking the input
will select and reveal the key. As soon as the key loses
selection, it is hidden again.
Test Plan: Project test.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D3270
Summary:
- Adds a dependency moment-guess (https://github.com/apoorv-mishra/moment-guess) to guess date formats from strings. However the npm package is missing source maps which leads to an ugly warning, so currently using a fork until https://github.com/apoorv-mishra/moment-guess/pull/22 is resolved.
- Adds guessDateFormat using moment-guess to determine the best candidate date format. The logic may be refined for e.g. lossless imports where the stakes are higher, but for now we're just trying to make type conversions smoother.
- Uses guessDateFormat to guess widget options when changing column type to date or datetime.
- Uses the date format of the original column when possible instead of guessing.
- Fixes a bug where choices were guessed based on the display column instead of the visible column, which made the guessed choices influenced by which values were referenced as well as completely broken when converting from reflist.
- @dsagal @georgegevoian This builds on https://phab.getgrist.com/D3265, currently unmerged. That diff was created first to alert to the change. Without it there would still be similar test failures/changes here as the date format would often be concretely guessed and saved as YYYY-MM-DD instead of being left as the default `undefined` which is shows as YYYY-MM-DD in the dropdown.
Test Plan: Added a unit test to `parseDate.ts`. Updated several browser tests which show the guessing in action during type conversion quite nicely.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: dsagal, georgegevoian
Differential Revision: https://phab.getgrist.com/D3264
Summary:
also:
- closes opened menu if any when click on a custom widget
- closes opened menu if any when F2
Test Plan: Include test case
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3269
Summary:
Enabled by default, the new checkbox is only visible to
users logged in with email/password, and controls whether it is possible
to log in to the same account via a Google account
(with matching email). When disabled, CognitoClient will refuse logins
from Google if a Grist account with the same email exists.
Test Plan:
Server and browser tests for setting flag. Manual tests to verify
Cognito doesn't allow signing in with Google when flag is disabled.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3257
Summary:
Available at login.getgrist.com/signup, the new sign-up page
includes similar options available on the hosted Cognito sign-up
page, such as support for registering with Google. All previous
redirects to Cognito for sign-up should now redirect to the new
Grist sign-up page.
Login is still handled with the hosted Cognito login page, and there
is a link to go there from the new sign-up page.
Test Plan: Browser, project and server tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3249
Summary:
Also fix few clode glitches
- Attach events handler to the scrollPane using `onMatch` instead of
the cell itself. This streamline cell dom creation a bit while
scrolling.
- Fix memory leaks on contextMenu.
- Also fix Importer and ColumnOps nbrowser test
see:
https://phab.getgrist.com/D3237#inline-36376https://phab.getgrist.com/D3237#inline-36375
Restore default context menu for where there's no custom one
It appears that default context menu adds some value, in particular
for links and for editing text. This diff restores it.
Test Plan: Should not break anything.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3256
Summary:
Custom widgets will receive all visible columns when they are not configured.
This will make the existing configuration work without the need to pick columns.
Test Plan: updated tests
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3259
Summary: Check that view exists first, otherwise fall back to default
Test Plan: Manual
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3260
Summary:
- Added a new special page for viewing raw data widgets:
- Implemented in DataTables.ts
- Accessible only via the special URL path `/p/data`
- Future diffs should make this page prettier and easily accessible
- Shows a list of user tables
- Clicking on a table name shows its `rawViewSection` by setting `GristDoc.viewModel.activeSectionId`. Note that in this case `GristDoc.viewModel` is an empty record, so this is a bit of a hack, but it works well and causes no known issues.
- Added `ViewSectionRec.isRaw` to know if the record represents a raw data widget.
- Added various restrictions in the UI for raw data widgets:
- 'Delete widget' is disabled in the 3-dot widget menu.
- Prevent hiding columns:
- "Hide column" in the column context menu is disabled
- The "VISIBLE/HIDDEN COLUMNS" section of the right panel > Table > Widget is hidden
- The toggle bar isn't configurable to ensure that users know when raw data is filtered:
- The filter bar always shows if and only if some filters are present
- "Toggle Filter Bar" is hidden in:
- Right panel > Table > Sort & Filter
- The sort/filter menu next to the three-dot menu for widgets.
- Other restrictions in the right panel:
- In the Column tab:
- 'Use separate settings' is disabled
- In the Table tab:
- In the Widget subtab:
- 'Change Widget' is hidden
- In the Data subtab:
- 'Edit Data Selection' is hidden
- 'SELECT BY' is hidden
Test Plan: Tested manually. The behaviour of raw data widgets may still change and they aren't easily visible to users yet.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3248
Summary:
Exposing new API in CustomSectionAPI for column mapping.
The custom widget can call configure method (or use a ready method) with additional parameter "columns".
This parameter is a list of column names that should be mapped by the user.
Mapping configuration is exposed through an additional method in the CustomSectionAPI "mappings". It is also available
through the onRecord(s) event.
This DIFF is connected with PR for grist-widgets repository https://github.com/gristlabs/grist-widget/pull/15
Design document and discussion: https://grist.quip.com/Y2waA8h8Zuzu/Custom-Widget-field-mapping
Test Plan: browser tests
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3241
Summary:
- Brings in a new utility `contextMenu` to open context menu next to the mouse position
- Use this utility to show a CellContextMenu, that sort of merge cell context menu and column context menu together.
- Show cell context menu on context click on any grid's cell.
- Also takes care of showing the row context menu for detail view on a context click that occurs on cells and not only on the row num header as it was the case prior to this diff.
- task: https://gristlabs.getgrist.com/doc/check-ins/p/5#a1.s9.r1529.c31
- discussion: https://grist.quip.com/ETGkAroLnc0Y/Cell-Context-Menu
{F40092}
Test Plan: - Adds project test and nbrowser for cell context menu and new cases for the detail row context menu.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D3237
Summary: Change code that conditionally depended on #newui in the URL to assume that everything is always in the new UI now.
Test Plan: this
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3246
Summary: This is https://phab.getgrist.com/D3205 plus some changes (https://github.com/dsagal/grist/compare/type-convert...type-convert-server?expand=1) that move the conversion process to the backend. A new user action ConvertFromColumn uses `call_external` so that the data engine can delegate back to ActiveDoc. Code for creating formatters and parsers is significantly refactored so that most of the logic is in `common` and can be used in different ways.
Test Plan: The original diff adds plenty of tests.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3240
Summary: Remove several unused methods in ViewConfigTab.js, and all of SummaryConfig.js.
Test Plan: this
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D3245
Summary:
When two widgets are linked by same-record linking, and the source of that link is also filter-linked, then it will pick up default values from its own filter-link source, but the same-record-link target didn't. This fixes that so that default values are filled in intuitively.
Moved the logic of linkingState, linkingFilter, and getDefaultColValues from BaseView.js to LinkingState.ts and ViewSectionRec.ts. In particular getDefaultColValues is now a property of LinkingState which may be copied from the source view section for a same-record link.
Note that `ViewSectionRec.linkingFilter` no longer uses `computerBuilder` and thus doesn't ignore dependencies inside LinkingState any more. I couldn't figure out how to make `linkingFilter` a `pureComputed` (otherwise I get recursion errors) that ignores dependencies. In any case, it's now important to have a dependency on `srcSection.linkingState()` for `getDefaultColValues` to work correctly, so I think this is for the best.
Test Plan: Added a new nbrowser test and fixture.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3238
Summary: The value seems unused
Test Plan: This is me testing if it's actually unused
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D3244
Summary:
Tweaks CSS of account page, ApiKey and MFAConfig to work better
on narrow-screen devices.
Test Plan: Tested manually.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3234
Summary:
Improving user experience on summary columns.
- Showing 'not-allowed' cursor on sections/menus that can't be changed
- Disabling menu options and buttons in the column behavior section that converts a formula column to a data column
- Showing nicer error message about converting formula to a data column.
Test Plan: manual tests, no behavior change
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3222
Summary:
Grist sessions created pre-MFA do not store access or
refresh tokens, which means that MFA status from Cognito
can't be loaded without requiring re-authentication. MFAConfig
handles this by requiring security verification as usual, and
checking if it needs to reload MFA status on success. If it does, it'll
close the 2FA setup dialog and reload, which should show the correct
2FA configuration status and buttons.
Test Plan: Updated existing tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3231
Summary:
Bumps aws-sdk to the latest version of v2.
Updates CognitoClient to use fake AWS credentials with service provider. While none
of our calls currently require real credentials, we still need to send fake credentials
with certain calls, such as AssociateSoftwareToken, to avoid getting errors back from
AWS about missing credentials.
Fixes some CSS and alignment issues with the authentication method tile buttons in
MFAConfig.
Test Plan: Existing Cognito tests. Manually tested login and 2FA setup still work.
Reviewers: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3228
Summary:
This fixes the cell color only applying to the content height of a
choice or reference list cell, and not the entire cell. Could
be reproduced by causing a row other than the choice/reference
list to grow in height.
Test Plan: Tested manually.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3226
