* Fix support of Scaleway S3 bucket #359
While MinIO and AWS return versionId as strings, other S3 API
implementations return versionId as integers.
We must carefully convert the versionId as string in order to cover
these various behaviors.
Also ensure that docStorage is initialized before attempting to
calculate the data size in order to avoid an exception.
* Add unit tests for MinIOExternalStorage#versions() #359
Introduced some unit tests to :
- ensure listObjects is called with the right arguments;
- cover the case when a S3 bucket implementation does not return the
versionId as a string but rather as an integer (like Scaleway):
in such a case, ensure that the returned snapshotId is a string;
- cover the case when the listObjects function emits an error, ensure the
versions() call rejets with the error emitted;
- that the deleteMarkers are only returned when the
includeDeleteMarkers is passed;
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary:
Remove a duplicate import, perhaps introduced during merge of
a relatively long-lived branch.
Test Plan: existing tests should pass
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3953
Summary:
introduces POST /api/docs/{docId}/webhooks and DELETE /api/docs/{docId}/webhooks/{webhookId} on place of old _subscribe and _unsubscribe endpoints.
Remove checking for unsubscribeKey while deleting webhook - only owner can delete webhook using DELETE endpoint. subscription key is still needed for _unsubscribe endpoint.
old _unsubscribe and _subscribe endpoints are still active and work as before - no changes there.
Posting schema:
```
POST /api/docs/[docId]/webhooks
```
Request Body:
```
{
"webhooks": [
{
"fields": {
"url": "https://webhook.site/3bd02246-f122-445e-ba7f-bf5ea5bb6eb1",
"eventTypes": [
"add",
"update"
],
"enabled": true,
"name": "WebhookName",
"memo": "just a text",
"tableId": "Table1"
}
},
{
"fields": {
"url": "https://webhook.site/3bd02246-f122-445e-ba7f-bf5ea5bb6eb2",
"eventTypes": [
"add",
],
"enabled": true,
"name": "OtherWebhookName",
"memo": "just a text",
"tableId": "Table1"
}
}
]
}
```
Expected response: WebhookId for each webhook posted:
```
{
"webhooks": [
{
"id": "85c77108-f1e1-4217-a50d-acd1c5996da2"
},
{
"id": "d87a6402-cfd7-4822-878c-657308fcc8c3"
}
]
}
```
Deleting webhooks:
```
DELETE api/docs/[docId]/webhooks/[webhookId]
```
there is no payload in DELETE request. Therefore only one webhook can be deleted at once
Response:
```
{
"success": true
}
```
Test Plan: Old unit test improved to handle new endpoints, and one more added to check if endpoints are in fact created/removed
Reviewers: alexmojaki
Reviewed By: alexmojaki
Subscribers: paulfitz, alexmojaki
Differential Revision: https://phab.getgrist.com/D3916
Summary:
Implements the latest design of the Formula AI Assistant.
Also switches out brace to the latest build of ace.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D3949
Test Plan: Added a check to the emoji test.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D3951
Grist by default uses node-sqlite3 to manipulate data in an
SQLite database. If a single parameter is passed to `run`
and it is a list, the list is unpacked and its contents treated
as the actual parameters. In grist-static, we use other SQLite
interfaces that don't have that automatic unpacking. Most
calls like this have been removed from Grist, but at least one
was missed, and was causing symptoms such as
https://github.com/gristlabs/grist-static/issues/5
This change should make no difference to regular Grist, but
resolves the grist-static problems.
Summary:
add back config call for external storage
in the right order.
Test Plan: existing saas tests catch this
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D3946
Summary:
Adding limits for AI calls and connecting those limits with a Stripe Account.
- New table in homedb called `limits`
- All calls to the AI are not routed through DocApi and measured.
- All products now contain a special key `assistantLimit`, with a default value 0
- Limit is reset every time the subscription has changed its period
- The billing page is updated with two new options that describe the AI plan
- There is a new popup that allows the user to upgrade to a higher plan
- Tiers are read directly from the Stripe product with a volume pricing model
Test Plan: Updated and added
Reviewers: georgegevoian, paulfitz
Reviewed By: georgegevoian
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3907
Summary:
In a selector table, when a selected row is filtered out of view, linked widgets should update based on the newly selected row.
There were a few bugs that contributed to this wrong behavior:
- Gridview wasn't subscribing to the current row id, and the row with id 'new' was being converted to the first row
- Cursor was keeping track of the currently selected row id, it was hiding a problem behind the proper rowIndex
- Undo/redo somehow leveraged the wrong rowId from the cursor during the position restore.
The `No data` text was also changed to be more meaningful.
Test Plan: Added and updated.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D3937
Summary:
A recent change broke the ability to open the account page if the current
org was inaccessible or invalid. Now the account page, and a few additional
pages, are excluded from needing an org to load.
Test Plan: Browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D3944
Summary:
It was impossible to focus the search input in the filter menu if either
of the range filter inputs were focused.
Test Plan: Browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D3942
Summary: Text in Document History > Activity tab is now selectable and one can copy it.
Test Plan: Manual
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D3939
Summary:
Adds a new Support Grist page (accessible only in grist-core), containing
options to opt in to telemetry and sponsor Grist Labs on GitHub.
A nudge is also shown in the doc menu, which can be collapsed or permanently
dismissed.
Test Plan: Browser and server tests.
Reviewers: paulfitz, dsagal
Reviewed By: paulfitz
Subscribers: jarek, dsagal
Differential Revision: https://phab.getgrist.com/D3926
Summary:
Here's a series of badness that easily leads to a crash, in reverse order:
- Lodash's map() function interprets an object with a .length property as an array.
- Some very old code generated human-friendly descriptions of user actions,
applying map() to parts of them. It so happens that this generated description
isn't even used.
- If a user action is encountered with a sufficiently large length propery,
map() would exhaust the server memory.
Fixed by removing old unneeded code, and replacing some other occurrences of
lodash's map() with native equivalents.
Test Plan: Tested manually on a local reproduction of the issue.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: paulfitz
Differential Revision: https://phab.getgrist.com/D3938
Summary:
- Detecting emoji is surprisingly tricky; we use a fancy regex as a decent heuristic.
- Icons are a little larger than before.
- Styling tweaked for light and dark modes
- In case the OS doesn't render the emoji as one character, truncate what's
shown in the icon box.
Test Plan: Added a test case.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D3904
Test Plan: Checked new looks manually, behavior should not be affected.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D3934
A small test harness bundle was recently added that is breaking the docker image build. It could be added to the docker image, but that would introduce a bunch of extraneous test file dependencies. So this tweaks the build to simply skip the test bundle if its primary source file is not found.
Also added some other test fixes along the way:
* make a custom widget test more reliable
* update a localization test now that `pt` exists
* store more log info in artifact on error
There's a little nest of SelectBy tests that sometimes fail.
They are the only tests with an import of a helper function from
an other file that contains tests. Such imports have caused trouble
with mocha in the past. I'm not sure if that is the case now, but
I'd like to eliminate it as a possibility.
Summary:
This tweaks the prompting so that the user's message is given on its own instead of as a docstring within Python. This is so that the prompt makes sense when:
- the user asks a question such as "Can you write me a formula which does ...?" rather than describing their formula as a docstring would, or
- the user sends a message that doesn't ask for a formula at all (https://grist.slack.com/archives/C0234CPPXPA/p1687699944315069?thread_ts=1687698078.832209&cid=C0234CPPXPA)
Also added wording for the model to refuse when the user asks for something that the model cannot do.
Because the code (and maybe in some cases the model) for non-ChatGPT models relies on the prompt consisting entirely of Python code produced by the data engine (which no longer contains the user's message) those code paths have been disabled for now. Updating them now seems like undesirable drag, I think it'd be better to revisit this when iteration/experimentation has slowed down and stabilised.
Test Plan:
Added entries to the formula dataset where the response shouldn't contain a formula, indicated by the value `1` for the new column `no_formula`.
This is somewhat successful, as the model does refuse to help in some of the new test cases, but not all. Performance on existing entries also seems a bit worse, but it's hard to distinguish this from random noise. Hopefully this can be remedied in the future with more work, e.g. automatic followup messages containing example inputs and outputs.
Reviewers: paulfitz
Reviewed By: paulfitz
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D3936
Summary:
This uses a newer version of mocha in grist-core so that tests can be run in parallel. That allows more tests to be moved without slowing things down overall. Tests moved are venerable browser tests; only the ones that "just work" or worked without too much trouble to are moved, in order to keep the diff from growing too large. Will wrestle with more in follow up.
Parallelism is at the file level, rather than the individual test.
The newer version of mocha isn't needed for grist-saas repo; tests are parallelized in our internal CI by other means. I've chosen to allocate files to workers in a cruder way than our internal CI, based on initial characters rather than an automated process. The automated process would need some reworking to be compatible with mocha running in parallel mode.
Test Plan: this diff was tested first on grist-core, then ported to grist-saas so saas repo history will correctly track history of moved files.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D3927
