For #1140, I considered trying to use the existing fields in a better
way, but because we already use the activations table to store
preferences, we need to keep all of the existing data and its usage
as-is.
The enterprise code will use this new column to decide how long the
trial period should be.
Summary:
When last widget for a table is removed, user is informed
about that and can decide between removing the widget and removing
both table and widget
Test Plan: Updated
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4295
Access control for ConvertFromColumn in the presence of access rules had previously been left as a TODO. This change allows the action when the user has schema rights. Because schema rights let you create formulas, they let you read anything, so there is currently no value in nuance here.
Summary:
A new onboarding page is now shown to all new users visiting the doc
menu for the first time. Tutorial cards on the doc menu have been
replaced with a new version that tracks completion progress, alongside
a new card that opens the orientation video.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4296
Summary: Forcing category xaxis type for bar chart when labels are not numerical.
Test Plan: Added new and updated existing
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4297
Summary:
Move an important set of tests that were in our SaaS
repo for no good reason.
Test Plan: moving tests
Reviewers: jordigh
Reviewed By: jordigh
Differential Revision: https://phab.getgrist.com/D4300
This adds a config file that's loaded very early on during startup.
It enables us to save/load settings from within Grist's admin panel, that affect the startup of the FlexServer.
The config file loading:
- Is type-safe,
- Validates the config file on startup
- Provides a path to upgrade to future versions.
It should be extensible from other versions of Grist (such as desktop), by overriding `getGlobalConfig` in stubs.
----
Some minor refactors needed to occur to make this possible. This includes:
- Extracting config loading into its own module (out of FlexServer).
- Cleaning up the `loadConfig` function in FlexServer into `loadLoginSystem` (which is what its main purpose was before).
Summary:
Some editors do some async work before saving the value (Ref column can add new
records). Those actions were send without bundling, so it wasn't possible to undo those
actions with togheter.
Test Plan: Added new test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4285
Summary:
- Adding confirmation dialog when user doesn't want to cancel site
- Changing `Cancel subscription` to `Cancel plan`
- Removing `Pro` from upgrade header on pricing modal
- Better handling situation when there is no default price
- Removing mentions about sprouts program
- Removing cache for stripe plans
Test Plan: Updated tests
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4273
Summary:
Version API endpoint wasn't logging telemetry from POST requests. The issue was in registration
order, this endpoint was registered before `expressJson` and it couldn't read json body in the handler.
Test Plan: Added new test
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4277
Summary:
- Fixing port allocation in TestServer
- Extending logging in the Billing test
- Fixing negative rowIds support for add/remove actions
- Making FormulaEditor and CardView tests less flacky
Test Plan: Existing
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz, dsagal
Differential Revision: https://phab.getgrist.com/D4280
Summary:
For non-owners, the timing section of Document Settings is now disabled.
For non-editors, the "Reload" section is disabled.
Test Plan: Added a test case for timing being disabled.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4275
Summary:
After release on 2024-06-12 (1.1.15) the GRIST_DEFAULT_PRODUCT env variable wasn't respected by the
method that started the server in single org mode. In all deployments (apart from saas), the default product
used for new sites is set to `Free`, but the code that starts the server enforced `teamFree` product.
This change adds a fix routine that fixes this issue by rewriting team sites from `teamFree` product to `Free`
product only if:
- The default product is set to `Free`
- The deployment type is something other then 'saas'.
Additionally there is a test that will fail after 2024.10.01, as this fix should be removed before this date.
Test Plan: Added test
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4272
This fixes the flaky test in "ViewLayoutCollapse.ts": "fix: should not dispose the instance when drag is cancelled".
The 'mouseenter' event wasn't consistently triggering properly on the drop target (LayoutEditor.ts - line 342) when the mouse was moved onto it.
The change simulates a "drag" over the drop target, moving the mouse into multiple positions over it, seemingly fixing the problem.
Summary:
Dropdown conditions can now reference a `user` variable, similar to the
one available in Access Rules.
Test Plan: Browser test.
Reviewers: jarek, paulfitz
Reviewed By: jarek, paulfitz
Differential Revision: https://phab.getgrist.com/D4255
Summary:
Clearing virtual tables after user navigates away from the pages
that show them. Leaving them behind will reveal them on the Raw
Data page, with a buggy experience as user can't view the data
there.
Test Plan: Extended tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: jarek, georgegevoian
Differential Revision: https://phab.getgrist.com/D4258
Summary:
A few tests that hadn't been ported to grist-core yet began failing after
a change in behavior with the column filter menu.
Test Plan: Existing tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4260
Summary:
Adding new buttons to control the `timing` API and a way to view the results
using virtual table features.
Test Plan: Added new
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4252
Summary:
- Reading plans from Stripe, and allowing Stripe to define custom plans.
- Storing product features (aka limits) in Stripe, that override those in db.
- Adding hierarchical data in Stripe. All features are defined at Product level but can be overwritten on Price levels.
- New options for Support user to
-- Override product for team site (if he is added as a billing manager)
-- Override subscription and customer id for a team site
-- Attach an "offer", an custom plan configured in stripe that a team site can use
-- Enabling wire transfer for subscription by allowing subscription to be created without a payment method (which is customizable)
Test Plan: Updated and new.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4201
Context:
On self-hosted instances, some places in the code rely on the fact that we resolves public domains while being behind reverse proxies. This leads to cases where features are not available, such as the "Duplicate document" one.
Bugs that are solved - n self-hosted instances:
Impossible to open templates and tutorials right after having converted them;
Impossible to submit forms since version 1.1.13;
Impossible to restore a previous version of a document (snapshot);
Impossible to copy a document;
Solution:
Introduce the APP_HOME_INTERNAL_URL env variable, which is quite the same as APP_DOC_INTERNAL_URL except that it may point to any home worker;
Make /api/worker/:assignmentId([^/]+)/?* return not only the doc worker public url but also the internal one, and adapt the call points like fetchDocs;
Ensure that the home and doc worker internal urls are trusted by trustOrigin;
---------
Co-authored-by: jordigh <jordigh@octave.org>
Summary: Conditional style rules can now be reordered by dragging and dropping them.
Test Plan: Browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4251
This adds support for $id in dropdown conditions, using the same
method used for supporting referencedColumn.id, and extends a test
to exercise the variable. Without this, the dropdown editor gives
an error if $id or rec.id is used, stating that the column is invalid.
Summary:
The Ace autocomplete sometimes doesn't appear if keys are entered too quickly.
A larger fixture document (World) used in the Importer2 tests sometimes takes longer than 5 seconds to import.
Test Plan: N/A
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4245
Summary:
The default value of Choice columns is empty string, but ChoiceEditor was
saving nulls whenever a blank value was saved. This was causing unexpected
updates to trigger values due to the cell value changing internally, even
though null and empty string appear the same in the UI.
Test Plan: Browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4242
Summary:
This adds a basic sandbox check to the admin panel. It also makes
the "probes" used in the boot page available from the admin panel,
though they are not yet displayed. The sandbox check is built as
a probe.
In the interests of time, a lot of steps had to be deferred:
* Reconcile fully the admin panel and boot page. Specifically, the
admin panel should be equally robust to common configuration problems.
* Add tests for the sandbox check.
* Generalize to multi-server setups. The read-out will not yet be useful
for setups where doc workers and home servers are configured
separately.
Test Plan: Added new test
Reviewers: jarek, georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4241
Summary:
Update for the admin page to show the latest available version information.
- Latest version is read from docs.getgrist.com by default
- It sends basic information (installationId, deployment type, and version)
- Checks are done only on the page itself
- The actual request is routed through the API (to avoid CORS)
Test Plan: Added new test
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4238
Summary:
Dropdown conditions let you specify a predicate formula that's used to filter
choices and references in their respective autocomplete dropdown menus.
Test Plan: Python and browser tests (WIP).
Reviewers: jarek, paulfitz
Reviewed By: jarek
Subscribers: dsagal, paulfitz
Differential Revision: https://phab.getgrist.com/D4235
Summary:
Renaming table after converting Ref column to RefList didn't work. During table rename, all Refs columns
are converted briefly to Int columns which treats values stored in RefList columns as errors, and stores its
`repr` strings. This could be recovered back if the value stored in RefList column was a plain list, but if we had there
a RecordList object, the RefList column didn't know how to parse that.
Test Plan: Added test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4233
Summary:
- /timing/start endpoint to start collecting information
- /timing/stop endpoint to stop collecting
- /timing to retrive data gatherd so far
Timings are collected for all columns (including hidden/helpers/system)
Test Plan: Added new
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4230
Summary:
This tightens down the set of referenced columns made available to
forms for dropdowns. Previous access to columns was computed at the
level of shared tables. Now it is calculated at the level of shared
sections. That means that we can now refrain from following hidden
references, and make the referred data unavailable to forms, since
they should not need it.
Test Plan: extended test
Reviewers: jarek, dsagal
Reviewed By: jarek, dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4234
Summary:
Stop maintaining a copy of redis stubs. Bring back product stub
for grist-ee.
Test Plan: existing tests should pass
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4232
Summary: The forms announcement popup should no longer be shown.
Test Plan: N/A
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4229
Summary:
Shares and documents would both produce a rule set for the same column
if the document rule set was for multiple columns. In this case, it was causing
one of the rules to be overwritten by the other (specifically, the rule granting
access to form references was not being applied in shares). The symptom was
`null` values in place of the referenced table's values.
We address this by splitting any rule sets for multiple columns that are also
affected by shares, so that they can be overridden by shares without causing a
conflicting rule set to be created (i.e. 2 column rule sets containing the same column).
Test Plan: Server tests.
Reviewers: dsagal, paulfitz
Reviewed By: dsagal, paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4208
Summary:
- New styling for forms.
- New field options for various field types (spinner, checkbox, radio buttons, alignment, sort).
- Improved alignment of form fields in columns.
- Support for additional select input keyboard shortcuts (Enter and Backspace).
- Prevent submitting form on Enter if an input has focus.
- Fix for changing form field type causing the field to disappear.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4223
Summary:
When one of 2 widget was collapsed, the resulting widget can become a root section. Then,
when a page was duplicated, the layout was duplicated incorrectly (with wrong collapsed
section). This resulted in a bug, when the root section was deleted, as it was the last
section in the saved layout, but not the last section on the visible layout.
Test Plan: Added 2 tests
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4227
Summary:
New endpoint `/api/version` that returns latest version of stable docker image in format:
```
{"latestVersion":"1.1.12","
updatedAt":"2024-03-06T06:28:25.752337Z","
isCritical":false,
"updateURL":"https://hub.docker.com/r/gristlabs/grist"
}
```
It connects to docker hub API and reads the version from the tag lists endpoint.
Stores telemetry passed from the client such us: current version, deployment type, installationId and others.
Test Plan: Added new test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4220
Summary:
The login email was being used for the "email" parameter of a billing endpoint, but
the endpoint was checking it against the non-login email when looking for a matching
profile.
Test Plan: Manual.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4226
* Shutdown Doc worker when it is not considered as available in Redis
* Use isAffirmative for GRIST_MANAGED_WORKERS
* Upgrade Sinon for the tests
* Run Smoke test with pages in English
* Add logic in /status endpoint
The motivation for supporting an alternative to WebSockets is that while all browsers supported by Grist offer native WebSocket support, some networking environments do not allow WebSocket traffic.
Engine.IO is used as the underlying implementation of HTTP long polling. The Grist client will first attempt a regular WebSocket connection, using the same protocol and endpoints as before, but fall back to long polling using Engine.IO if the WebSocket connection fails.
Include these changes:
- CORS websocket requests are now rejected as a stronger security measure. This shouldn’t affect anything in practice; but previously it could be possible to make unauthenticated websocket requests from another origin.
- GRIST_HOST variable no longer affects CORS responses (also should not affect anything in practice, as it wasn't serving a useful purpose)
Summary:
- Add InstallAdmin class to identify users who can manage Grist installation.
This is overridable by different Grist flavors (e.g. different in SaaS).
It generalizes previous logic used to decide who can control Activation
settings (e.g. enable telemetry).
- Implement a basic Admin Panel at /admin, and move items previously in the
"Support Grist" page into the "Support Grist" section of the Admin Panel.
- Replace "Support Grist" menu items with "Admin Panel" and show only to admins.
- Add "Support Grist" links to Github sponsorship to user-account menu.
- Add "Support Grist" button to top-bar, which
- for admins, replaces the previous "Contribute" button and reopens the "Support Grist / opt-in to telemetry" nudge (unchanged)
- for everyone else, links to Github sponsorship
- in either case, user can dismiss it.
Test Plan: Shuffled some test cases between Support Grist and the new Admin Panel, and added some new cases.
Reviewers: jarek, paulfitz
Reviewed By: jarek, paulfitz
Differential Revision: https://phab.getgrist.com/D4194
Summary:
- Forms now have a reset button.
- Choice and Reference fields in forms now have an improved select menu.
- Formula and attachments column types are no longer mappable or visible in forms.
- Fields in a form widget are now removed if their column is deleted.
- The preview button in a published form widget has been replaced with a view button. It now opens the published form in a new tab.
- A new share menu for published form widgets, with options to copy a link or embed code.
- Forms can now have multiple sections.
- Form widgets now indicate when publishing is unavailable (e.g. in forks or unsaved documents).
- General improvements to form styling.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4203
Summary:
- Also move out reused helper methods out of test suites, to avoid inadvertently including some test suites into other suites.
- Add logging around onNewTab since that seems related to test failures.
- Skip several tests in Billing that fail particularly often.
Test Plan: Checking that tests pass, and how many. Compared to the last successful run on master, there are 4 new skips (this diff), and 3 fewer test cases (a test case run in triplicate was removed in https://github.com/gristlabs/grist-core/pull/899)
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4216
Summary:
If a table T contains a RefList:T showing RowId and there
are any empty cells, the table can't be removed.
Test Plan: Added new test
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4215
Summary: Adds support for importing .dsv files (an April Fools 2024 easter egg), and options for exporting .dsv and .tsv files from the Share menu.
Test Plan: Browser and server tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4210
Summary:
1. Set pageLoad timeout to 10s (default is 5 minutes)
2. Disable chrome's prompts to save credit card info, which may be
affecting Stripe pages
3. Periodically record of logs and screenshots for the most-failing test case
so that whenever it fails in a bad way (timeout with no indication of
what's wrong), we can hope to find out what's wrong.
Test Plan: Planning to celebrate if Billing tests pass.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4193
Summary: Using the lookup shortcut in the add column menu to find a max or min value is a buggy experience. MAX and MIN returns 0 for empty collections which can be interpreted as 1970.1.1 in date or date time columns. It is better to just return None (empty cell) when there are no records.
Test Plan: Updated
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4206
This check should be unnecessary for stores with strong consistency guarantees (virtually everywhere now).
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary: Empty doc tours are now skipped. Before, an error was shown instead.
Test Plan: Browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4204
Summary:
- UI now trims column rules before saving.
- When rules are loaded, bits that aren't applicable to a resource get ignored.
This should fix the incorrect behavior in existing docs without a migration.
Test Plan:
- Added test of UI, that it now sends trimmed rules
- Added a unitteset of new trimPermissions() function
- Add test of fixed interpretation of existing rules: now only permission bits
applicable to a resource get respected. I.e. create/delete/schemaEdit are
ignored in column rules, and schemaEdit is also ignored in table rules.
- Note that DuplicateTest was affected: updated on the assumption that
schemaEdit still can't actually apply at a table level.
Reviewers: georgegevoian, paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4205
This is a start at a page for diagnosing problems while setting up Grist. Starting to add some diagnostics based on feedback in github issues. We should make Grist installation easier! But when there is a problem it should be easier to diagnose than it is now, and this may help. The page is ugly and doesn't have many diagnostics yet, but we can iterate.
Visit `/boot` on a Grist server for tips on how to use this feature.
Summary: Showing configuration screen when widget is not mapped
Test Plan: New test added
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4192
One particular RawData test failure looks to be a consequence of notification toasts building up enough to cover a strategic button. This makes a somewhat simple-minded and brutal fix.
Also gives RemoveTransformColumns a bit more time.
Summary:
There has been inconsistency in using display email vs normalized email, which
ends up creating some duplication in downstream analyses (e.g. the same user
showing up twice with different capitalization).
1. Add UserProfile.loginEmail field with normalized email to prefer, when set, over the inconsistently used UserProfile.email.
2. In one place where it's not available, normalize the display email manually.
3. Clean up some code in Client.ts.
Unrelated tweak to API Console to be clear when a URL parameter wasn't found (rather than show whatever happens to be the first value).
Several test robustness improvements:
- Misplaced parenthesis in gristWebDriverUtils has been causing optTimeout argument to be ignored in tests, and treated always as indefinite.
- Attempt to fix SortMenu test by ignoring (retrying with logging) errors in waitForServer, which include "script timeout" errors that come from a non-configurable selenium or chromedriver timeout.
- Attempt to improve onNewTab() helper, which plays a role in failing Billing tests.
Test Plan: Tested manually the capitalization of logged emails. Counting on existing tests to catch issues.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4188
Summary:
Fixes misc. bugs with forms, updates Grist URLs on static form pages to link
to the new forms marketing page, and adds a forms announcement popup that's
shown next to the Add New button within a document.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4185
Summary:
few things is going here:
1. Added comand that can be called with intent to show behavioral popup somewhere else. I've added it to trigger showing reference popup from new colum menu, despite popup existing in FieldBuilder
2. Command for showing right panel get an argument to switch for choosen tab right after showing.
Test Plan: two test added - one to check if right side menu is shown after chosing ref or refList, and second to check if popup is shown.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4138
Summary:
If the longer OpenAI model exceeds the OpenAPI context length, we now perform another retry with a
shorter variant of the formula prompt. The shorter prompt excludes non-referenced tables and lookup
method definitions, which should help reduce token usage in documents with larger schemas.
Test Plan: Server test.
Reviewers: JakubSerafin
Reviewed By: JakubSerafin
Subscribers: JakubSerafin
Differential Revision: https://phab.getgrist.com/D4184
Summary:
It was manifesting as error "docactions.[Bulk]UpdateRecord for
non-existent record #1", and due to manualSort column having internal
state that wasn't properly cleared.
Test Plan: Added a test case that fails without the fix.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4183
Summary:
Adds a new test for formatting and fix several related bugs it uncovered:
1. When editing a number with "," decimal separator, ensure it opens in
the editor with "," (rather than ".", the original bug motivating this).
2. When guessing number format, set maxDecimals when it's needed
(otherwise, e.g. "$1.234", or "4.5%" weren't guessed as numeric)
3. When guessing number format, ignore whitespace when deciding if
guessed format is correct (otherwise percents can't be guessed in
locales which add "%" with a non-breaking space before it).
Test Plan: Added a test case that exercises all fixed behaviors.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4177
Summary:
Login (and other) middleware was included in the public form URL by mistake,
forcing logins on forms hosted on non-personal sites.
Test Plan: Browser test.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4181
