Summary: to allow custom widget having optional translations, lagunage seeted in user profile is passed as query parameter to custom widget
Test Plan: test added to check if query parameter is existing in url when settings is changed in profile
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: jarek, paulfitz
Differential Revision: https://phab.getgrist.com/D4045
Summary:
- /status accepts new optional query parameters: db=1, redis=1, and timeout=<ms> (defaults to 10_000).
- These verify that the server can make trivial calls to DB/Redis, and that they return within the timeout.
- New HealthCheck tests simulates DB and Redis problems.
- Added resilience to Redis reconnects (helped by a test case that simulates disconnects)
- When closing Redis-based session store, disconnect from Redis (to avoid hanging tests)
Some associated test reorg:
- Move stripeTools out of test/nbrowser, and remove an unnecessary dependency,
to avoid starting up browser for gen-server tests.
- Move TcpForwarder to its own file, to use in the new test.
Test Plan: Added a new HealthCheck test that simulates DB and Redis problems.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4054
Summary:
The new Command API provides limited access to Grist Commands from within cusotm
widgets. This includes the ability to perform undo and redo, which is bound to
the same keyboard shortcut as Grist by default.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: paulfitz, jarek
Differential Revision: https://phab.getgrist.com/D4050
Summary:
Fixes a very specific bug reported here: https://grist.slack.com/archives/C069RUP71/p1694630242765769
The error occurred when:
1. Removing multiple columns simultaneously
2. Those columns were sources of groupby columns for a summary table (so removing them meant recreating the summary table and thus deleting its columns)
3. There was a display column for one of the columns that got deleted (either directly or indirectly) which was set to be automatically removed since it was no longer needed, but this failed because the column was already deleted as part of earlier table removal.
I fixed this by making `apply_auto_removes` remove table records last, so removing the display column wouldn't be a problem.
That fixed the original error, but then I noticed that trying to undo the removal could lead to another error (under even more specific circumstances). It's hard to see exactly why, but I can see that just 3 `RemoveColumn` user actions generated over 100 doc actions and corresponding undo actions, hence the difficulty in narrowing the problem down. This is partly because removing a single column would recreate a summary table, only for that table to be immediately replaced again when another column was removed. Making the frontend send a single `[BulkRemoveRecord, _grist_Tables_column, ...] ` leads to a more efficient and sensible process with about half as many doc actions and no undo error. I think this alone would also solve the original error, but the data engine change seems more generally helpful and worth keeping.
Test Plan: Added a Python test and an nbrowser test
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4052
Test Plan: Added a test case that reproduces the bug and tests the fix
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: alexmojaki
Differential Revision: https://phab.getgrist.com/D4057
Summary:
Adding new "Delete my account" button to the profile page that allows users to remove completely
their accounts as long as they don't own any team site.
Test Plan: Added
Reviewers: georgegevoian, paulfitz
Reviewed By: georgegevoian, paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4037
Summary:
On mobile, tips for calendar and card list aren't currently
shown, but the creator panel was still automatically being
opened in preparation for showing the tip.
Test Plan: Manual and existing tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D4053
Summary:
Add two shutdown-related timeouts.
1. One is to limit the duration of any work that happens once shutdown
begins. In particular, waiting for an update to current time could block
indefinitely if the data engine is unresponsive. Such awaits are now
limited to 5 seconds.
2. The other is to allow documents to get shutdown for inactivity even when
some work takes forever. Certain work (e.g. applying user actions)
generally prevents a document from shutting down while it's pending. This
prevention is now limited to 5 minutes.
Shutting down a doc while something is pending may break some
assumptions, and lead to errors. The timeout is long to let us assume
that the work is stuck, and that errors are better than waiting forever.
Other changes:
- Periodic ActiveDoc work (intervals) is now started when a doc finishes
loading rather than in the constructor. The difference only showed up in
tests which makes the intervals much shorter.
- Move timeoutReached() utility function to gutil, and use it for
isLongerThan(), since they are basically identical. Also makes sure that the
timer in these is cleared in all cases.
- Remove duplicate waitForIt implementation (previously had a copy in both
test/server and core/test/server).
- Change testUtil.captureLog to pass messages to its callback, to allow asserts
on messages within the callback.
Test Plan:
Added new unittests for the new shutdowns, including a replication
of a bad state that was possible during shutdown.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D4040
Allows bidirectional / cyclic linking for same-record cursor links.
This should allow multiple sections to all synchronize their cursors,
such that clicking in any one of them will move all the others.
Works even if some sections in the cycle have rows filtered out (the
filtered-out sections might desync their cursors, but the correct cursor
position will still propagate downstream, and they'll re-sync if clicking on
a row that is present in them)
Under the hood, each cursor has a _lastEditedAt counter, updated when
a user's action changes the cursor in a section, such that we can always
tell which section was touched most recently. This is used to resolve
conflicts stably when dealing with cycles or chains of cursor-links.
Updated selectBy and recursiveMoveToCursorPos to handle cycles
Updated tests for selectBy behavior
However, main bidirectional-linking tests are not in this commit, they'll come in a subsequent PR
Summary: Fixing flakiness in the test helper.
Test Plan: Existing
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4051
Summary:
Polishes support for dark mode and enables syncing with the OS theme
by default.
Test Plan: Manual.
Reviewers: JakubSerafin
Reviewed By: JakubSerafin
Subscribers: JakubSerafin
Differential Revision: https://phab.getgrist.com/D4041
Summary:
In non-SaaS Grist, the AppHeader will no longer link to templates
page. Also, the last visited site is now reset when switching accounts.
Test Plan: Existing tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4048
Summary:
The tip is shown in the creator panel, in the subtab that lists the column
mapping configuration for the calendar widget. The panel now automatically
opens the first time a calendar widget is added to a page (via the Add New menu).
Test Plan: Manual.
Reviewers: JakubSerafin
Reviewed By: JakubSerafin
Subscribers: JakubSerafin, jarek
Differential Revision: https://phab.getgrist.com/D4047
Summary:
Simple click (a click on a already focused cell) was opening the editor
even though user clicked an anchor in the cell.
Test Plan: Added test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4044
Summary: Mapping for a single column in custom widgets accepts now multiple types as comma separeted list.
Test Plan: Added new
Reviewers: JakubSerafin
Reviewed By: JakubSerafin
Differential Revision: https://phab.getgrist.com/D4042
Summary:
In browser where user is logged in multiple times, she can start
creating a new site from docs org where user B is the default one, and end up on the
api endpoint where user A is the default one. This resulted in wrong userId in the handler.
Test Plan: New test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4029
Summary:
Previously, when a table was deleted, ref/reflist columns pointing at that table were deleted as well. This diff changes that, converting the columns to a suitable type instead.
See here for discussion and decisions: https://grist.slack.com/archives/C069RUP71/p1686060034848139
Test Plan: Added Python tests for most cases, and a DocApi test for where Python has to call JS code to convert the values.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4011
* Linkingstate Refactor, and displaying link info in rightpanel
Big refactor to LinkingState
Collects descriptive/user-facing labels into FilterState
Unifies/cleans up some logic
Adds LinkTypeDescription, a string enum which can be used
to easily switch/case between various cases of linking, and
codifies the logic in one place (currently only used for linkInfo)
Adds Link info to creator panel, near SelectBy dropdown
Bugfix: Disables linking from Attachment columns
Bugfix/Behavior change: changed linking with empty RefLists to better
match behavior of refs.
for context: Linking by a blank Ref filters to show records with a
blank value for that Ref. Previously this didn't work with RefLists.
Linking from a blank refList would show no records
(except in some cases involving summary tables)
Fixed this so that linking by a blank val consistently means "show
all records where the corresponding col is blank"
* `GRIST_ANON_PLAYGROUND`: When set to 'false' deny anonymous users access to the home page
* `GRIST_FORCE_LOGIN`: Much like `GRIST_ANON_PLAYGROUND` but don't support anonymous access at all (features like sharing docs publicly requires authentication)
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary:
Duplicate document popup wasn't offering workspace selection for personal orgs.
Also, when workspace was removed, the URL wasn't updated which left user with an error page `workspace not found`.
Test Plan: Updated
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4031
Summary:
The new "copyDoc" query parameter on the login page sets a short-lived cookie, which is
then read when welcoming a new user to copy that document to their Home workspace, and
redirect to it. Currently, only templates and bare forks set this parameter.
A new API endpoint for copying a document to a workspace was also added.
Test Plan: Browser tests.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3992
* add an endpoint for doing SQL selects
This adds an endpoint for doing SQL selects directly on a Grist document. Other kinds of statements are not supported. There is a default timeout of a second on queries.
This follows loosely an API design by Alex Hall.
Co-authored-by: jarek <jaroslaw.sadzinski@gmail.com>
Summary:
Iframe with custom widget is marked with a test class `test-custom-widget-ready` when
it receives the `ready` message from the rendered widget.
Test Plan: Added and updated. Existing test should pass.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4023
Summary: Added boilerplate code needed to create new wigets in "Add new" menu, that are wrapped around existing custom widgets. More details can be found here: https://grist.quip.com/larhAGRKyl6Z/Custom-widgets-in-Add-Widget-menu
Test Plan: nbowser tests added to verify if item in menu exits, if widget is rendered, and right side menu has widget selection and read access selection hided.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3994
Every fetch made from the client is logged to the console.
But this isn't really necessary, and is particularly confusing
in grist-static, where those fetches are virtualized.
Tests in grist-saas may need adjusting to remove the logger.
Summary:
Adding a new method `setCursorPos` in the widget API, and a new configuration option for the ready message `allowSelectBy` that exposes custom widgets in the `Select by` dropdown.
With this, a custom widget can control the position of the linked widgets and is able to change the column in the creator panel.
Test Plan: Added new test. Existing tests should pass.
Reviewers: JakubSerafin
Reviewed By: JakubSerafin
Subscribers: JakubSerafin
Differential Revision: https://phab.getgrist.com/D3993
Summary:
- Using a sample of data was causing poor detection if the sample were
cut mid-character. Switch to using line-based detection.
- Add a simple option for changing encoding. No convenient UI is offered
since config UI is auto-generated, but this at least makes it possible to
recover from bad guesses.
- Upgrades chardet library for good measure.
- Also fixes python3-building step, to more reliably rebuild Python
dependencies when requirements3.* files change.
Test Plan:
Added a python-side test case, and a browser test that encodings can
be switched, errors are displayed, and wrong encodings fail recoverably.
Reviewers: alexmojaki
Reviewed By: alexmojaki
Differential Revision: https://phab.getgrist.com/D3979
When an ACL formula fails to be run, a warning is printed. However, it is painful to know which formula is concerned by the warning.
ACL: if RHS is null, return false for "in" and "not in"
Summary:
move a test of access control formulas
to grist-core
Test Plan: moving a test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4013
Summary:
Moves assistant test to core.
Fixes new name for constaint.
Works around a chai-as-promised issue.
Test Plan: this
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4010
This adds an ASSISTANT_CHAT_COMPLETION_ENDPOINT which can be used
to enable AI Assistance instead of an OpenAI API key. The assistant
then works against compatible endpoints, in the mechanical sense.
Quality of course will depend on the model. I found some tweaks
to the prompt that work well both for Llama-2 and for OpenAI's models,
but I'm not including them here because they would conflict with some
prompt changes that are already in the works.
Co-authored-by: Alex Hall <alex.mojaki@gmail.com>
Summary:
{F74398}
Refactored the 'radio checkboxes' in the modal for deleting a page and reused them here.
The option to download as a template already existed in the server code but wasn't being exercised by the frontend. Also added an option to remove just the history, which is the main motivation for this diff.
Test Plan: Expanded the existing nbrowser test.
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D3999
Summary:
Sometimes the model repeats the classes given in the prompt which would mess up extracting the actual formula. This diff solves this by:
1. Changes the generated Python schema so that (a) the thing that needs completing is a plain top level function instead of a property/method inside the class and (2) the classes are fully valid syntax, which makes it easier to
2. Remove classes from the parsed Python code when converting the completion to a formula.
3. Tweak the prompt wording to discourage including classes in general, especially because sometimes the model tries to solve the problem by defining extra methods/attributes/classes.
While I was at it, I changed type hints to use builtins (e.g. `list` instead of `List`) to prevent `from typing import List` which was happening sometimes and would look weird in a formula. Similarly I removed `@dataclass` since that also implies an import, and this also fits with the tweaked wording that the classes are fake.
Test Plan:
Added a new test case to the formula dataset which triggers the unwanted behaviour. The factors that seem to trigger the problem are (1) a small schema so the classes are easier to repeat and (2) the need to import modules, which the model wants to place before all other code. The case failed before this diff and succeeded after. The tweaked wording reduces the chances of repeating the classes but didn't eliminate it, so forcibly removing the classes in Python was needed.
There were also a couple of other existing cases where repeating the classes was observed before but not after.
Overall the score increased from 49 to 51 out of 69 (including the new case). At one point the score was 53, but changes in whitespace were enough to make it drop again.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4000
