Commit Graph

143 Commits

Author SHA1 Message Date
George Gevoian
3aeff3c547 (core) Downgrade typeorm to 0.3.17
Summary:
SaaS bundling still uses browserify, which can't handle bundling typeorm
0.3.18 and beyond.

Also fixes an unrelated, flaky Notification test.

Test Plan: Existing tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4243
2024-04-30 16:13:33 -04:00
Jordi Gutiérrez Hermoso
61cb80d4e3 (core) Security updates for package.json following version bump update from grist-core
Summary: Fix the build and tests following version bump from [#921](https://github.com/gristlabs/grist-core/pull/921) from grist-core

Test Plan: Existing tests pass

Reviewers: georgegevoian, jordigh

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4239
2024-04-29 18:50:56 -04:00
fflorent
b04304342d Strict version in package.json 2024-04-29 14:54:36 -04:00
fflorent
9bacfc8287 Use chokidar to bundle css files 2024-04-29 14:54:36 -04:00
fflorent
14061fd5ac Remove catw 2024-04-29 14:54:36 -04:00
fflorent
f405ae715b Bump dependencies versions 2024-04-29 14:54:36 -04:00
Jordi Gutiérrez Hermoso
5753d07980 v1.1.13 2024-04-15 13:39:09 -04:00
CamilleLegeron
fe9cc80ccc
Create team site for self-hosted instances (#903) 2024-04-15 00:55:57 -07:00
Florent
4a9b6fea9d
Shutdown Doc worker when it is not considered as available in Redis #831 (#856)
* Shutdown Doc worker when it is not considered as available in Redis
* Use isAffirmative for GRIST_MANAGED_WORKERS
* Upgrade Sinon for the tests
* Run Smoke test with pages in English
* Add logic in /status endpoint
2024-04-04 10:25:42 -04:00
Jonathan Perret
96b652fb52
Support HTTP long polling as an alternative to WebSockets (#859)
The motivation for supporting an alternative to WebSockets is that while all browsers supported by Grist offer native WebSocket support, some networking environments do not allow WebSocket traffic.

Engine.IO is used as the underlying implementation of HTTP long polling. The Grist client will first attempt a regular WebSocket connection, using the same protocol and endpoints as before, but fall back to long polling using Engine.IO if the WebSocket connection fails.

Include these changes:
- CORS websocket requests are now rejected as a stronger security measure. This shouldn’t affect anything in practice; but previously it could be possible to make unauthenticated websocket requests from another origin.
- GRIST_HOST variable no longer affects CORS responses (also should not affect anything in practice, as it wasn't serving a useful purpose)
2024-03-28 13:22:20 -04:00
George Gevoian
cf6579dffa v1.1.12 2024-03-05 14:51:18 -05:00
George Gevoian
6b0ef41255 v1.1.11 2024-02-06 11:00:01 -05:00
Dmitry S
11afc08f65 (core) Improve API Console and link from Document Settings.
Summary:
Changes to building and serving:
- Remove unpkg dependencies, add npm module for swagger-ui-dist instead.
- Move apiconsole JS logic into core/app/client/apiconsole.ts, and use TypeScript.
- Add symlinks to swagger in static/ and core/static/.
- Refactor loadScript, and add loadCssFile; use these to load swagger-ui resources.

Changes to console itself:
- Support docId, workspaceId, orgId URL parameters. When present, the matching
  value in dropdowns is moved to the front and marked as "(Current)".
- Fix the ordering of example values, particularly for workspaces.
- Remove unwanted example values.
- Hide confusing "Authorize" button.
- Hide API keys, and rely consistently on cookies for executing API calls.

Integration into Grist:
- Added a button to Document Settings, just under document ID in "API".
- The button opens a separate page, passing in org, workspace, and doc info for the current doc.

Test Plan: Only tested manually, no automated tests yet.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4173
2024-01-29 10:08:19 -05:00
Dmitry S
be0b4a1968 (core) Use slugify module to construct slugs in urlIds, addressing a long-standing TODO.
Summary:
This offers better handling for accented and non-English characters,
e.g. "Événements" becomes "Evenements" (rather than "vnements") and
"таблиця" becomes "tablicya" rather than an empty string.

Test Plan:
Added a test case. Existing documents will auto-redirect to
newly-generated names. In cases where it's different, that's probably for the
best.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4176
2024-01-29 10:02:27 -05:00
Paul Fitzpatrick
4378ec24d8
v1.1.10 2024-01-03 07:57:53 -05:00
Paul Fitzpatrick
e0d44eff1f
set TEST_CLEAN_DATABASE when running server tests (#806)
After adding a batch of new server tests, some interactions between
tests have shown up via a shared database. This sets an existing flag
for dealing with this problem, that is used during browser tests but
hadn't been needed before for server tests.
2023-12-27 09:56:59 -05:00
Paul Fitzpatrick
145138b7e9 (core) move apiserver tests to core, disentangling notifier+billing parts
Summary:
This moves some more tests to core that would be useful for ANCT,
which had been stuck in grist-saas due to some entanglements with
sendgrid and billing. For sendgrid, I've moved around just enough
material to permit the tests to run mostly unchanged. Ideally
the interface to a notification system would be generalized, but
that's a bigger project.

Test Plan:
checked that tests are likely to run as expected
in core using preview laid out by ./buildtools/build_core.sh

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4149
2023-12-26 08:43:12 -05:00
Jarosław Sadziński
a424450cbe (core) Forms feature
Summary:
A new widget type Forms. For now hidden behind GRIST_EXPERIMENTAL_PLUGINS().
This diff contains all the core moving parts as a serves as a base to extend this functionality
further.

Test Plan: New test added

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4130
2023-12-20 13:23:12 +01:00
Jarosław Sadziński
1116f2d227 (core) Updating dependencies
Summary: Updating npm packages in preparation for forms feature

Test Plan: Existing

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4142
2023-12-12 20:05:24 +01:00
Paul Fitzpatrick
1ccad9171e (core) updates from grist-core 2023-12-11 09:07:35 -05:00
Dmitry S
09c84734db (core) Enable the 'none' option in ColorSelect for cell and header text styles
Test Plan: Added a test case

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4120
2023-12-04 19:13:52 -05:00
George Gevoian
c64f64b11c v1.1.9 2023-12-01 16:34:41 -05:00
Paul Fitzpatrick
de13a2fd7a
include the @gristlabs/grist-widget package, so Calendar is always available (#745)
The Calendar feature was implemented as a custom widget. To make it
available offline, we prepare a package that includes it, and add that
to Grist. The PluginManager is configured to find it.

An optional `GRIST_SKIP_BUNDLED_WIDGETS` flag is added to disable
widgets bundled this way from being used. This may be needed by
the tests in grist-widget to avoid getting an echo :-)
2023-11-28 09:28:15 -05:00
Florent
e3cc76b67d
Force language to English while running the tests (#776)
While running the tests in an environment whose default locales are not
English, the tests fails. This is due to the fact that the webpages
rendered in the nbrowsers tests are expected to be in English.

Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-11-27 06:26:11 -05:00
George Gevoian
764775d104 v1.1.8 2023-11-20 10:20:13 -05:00
Jarosław Sadziński
dda1b5cf1b (core) Fixing packages after merge with core
Summary: Reconciling package.json between core and hosted version.

Test Plan: Existing

Reviewers: JakubSerafin

Reviewed By: JakubSerafin

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4115
2023-11-14 13:34:18 +01:00
Florent
a4998b4b21
Add native OIDC support in Grist #707 (#717) 2023-11-09 14:56:39 -05:00
George Gevoian
ded6772256 v1.1.7 2023-11-06 10:43:50 -05:00
Florent
38c8476aff
Add option to serve Prometheus metrics #671 (#693)
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-10-24 17:36:53 -04:00
Paul Fitzpatrick
b63b042390
v1.1.6 2023-10-19 01:23:37 +02:00
George Gevoian
f1cf92aca1 (core) Polish new Add Column menu
Summary: Fixes and features for the unreleased Add Column menu.

Test Plan: Manual.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D4076
2023-10-17 15:39:53 -04:00
George Gevoian
0cadb93d25 (core) Update dependencies
Summary:
Changes the minimum version of Node to 18, and updates the Docker images and GitHub workflows to build Grist with Node 18.

Also updates various dependencies and scripts to support building running tests with arm64 builds of Node.

Test Plan: Existing tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3968
2023-10-11 17:36:58 -04:00
Paul Fitzpatrick
97a84ce6ee
v1.1.5 2023-10-04 09:43:59 -04:00
George Gevoian
f38df564a9 (core) Add Command API to Grist Plugin API
Summary:
The new Command API provides limited access to Grist Commands from within cusotm
widgets. This includes the ability to perform undo and redo, which is bound to
the same keyboard shortcut as Grist by default.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: paulfitz, jarek

Differential Revision: https://phab.getgrist.com/D4050
2023-09-27 13:25:18 -04:00
Paul Fitzpatrick
a0f5ab81ad
v1.1.4 2023-09-08 10:13:39 -04:00
Dmitry
c9489e4a61
Fix ManyFetches test, and include into grist-core a needed upgrade to the 'ws' package. (#614)
This is a follow-up fix to 526a5df157.
2023-08-07 18:39:14 -04:00
Paul Fitzpatrick
08e7f24ff8
v1.1.3 2023-08-03 11:52:58 -04:00
George Gevoian
ea8a59c5e9 (core) Implement AI Assistant UI V2
Summary:
Implements the latest design of the Formula AI Assistant.

Also switches out brace to the latest build of ace.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3949
2023-07-13 10:30:35 -04:00
Florent
a56b0448ff
Fix DEBUG not accepting numbers #559 (#561)
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-07-07 18:17:31 -04:00
Paul Fitzpatrick
37bdaccac6
v1.1.2 2023-07-04 07:23:04 -04:00
Paul Fitzpatrick
fc16b4c8f6 (core) updates from grist-core 2023-06-27 03:04:42 -04:00
Paul Fitzpatrick
bcbf57d590 (core) bump mocha version to allow parallel tests; move more tests to core
Summary:
This uses a newer version of mocha in grist-core so that tests can be run in parallel. That allows more tests to be moved without slowing things down overall. Tests moved are venerable browser tests; only the ones that "just work" or worked without too much trouble to are moved, in order to keep the diff from growing too large. Will wrestle with more in follow up.

Parallelism is at the file level, rather than the individual test.

The newer version of mocha isn't needed for grist-saas repo; tests are parallelized in our internal CI by other means. I've chosen to allocate files to workers in a cruder way than our internal CI, based on initial characters rather than an automated process. The automated process would need some reworking to be compatible with mocha running in parallel mode.

Test Plan: this diff was tested first on grist-core, then ported to grist-saas so saas repo history will correctly track history of moved files.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3927
2023-06-27 02:55:34 -04:00
Florent
bf15a14dd4
Add command to debug the server (#533)
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-06-20 12:49:57 -04:00
Florent
fa306eba15
Support spaces in GREP_TESTS values (#535)
Not only GREP_TESTS can be assigned a single word like:
GREP_TESTS=DocApi yarn test

But also can be assigned a whole sentence part:
GREP_TESTS="supports ascending sort" yarn test

That's especially useful to run a single test (and not a whole test
suit)

Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-06-15 15:45:32 -04:00
Paul Fitzpatrick
0fb5092e33 (core) updates from grist-core 2023-06-05 08:33:28 -04:00
Paul Fitzpatrick
cd51eee160
v1.1.1 2023-06-02 14:15:34 -04:00
Dmitry S
d191859be7 (core) For exporting XLSX, do it memory-efficiently in a worker thread.
Summary:
- Excel exports were awfully memory-inefficient, causing occasional docWorker
  crashes. The fix is to use the "streaming writer" option of ExcelJS
  https://github.com/exceljs/exceljs#streaming-xlsx-writercontents. (Empirically
  on one example, max memory went down from 3G to 100M)
- It's also CPU intensive and synchronous, and can block node for tens of
  seconds. The fix is to use a worker-thread. This diff uses "piscina" library
  for a pool of threads.
- Additionally, adds ProcessMonitor that logs memory and cpu usage,
  particularly when those change significantly.
- Also introduces request cancellation, so that a long download cancelled by
  the user will cancel the work being done in the worker thread.

Test Plan:
Updated previous export tests; memory and CPU performance tested
manually by watching output of ProcessMonitor.

Difference visible in these log excerpts:

Before (total time to serve request 22 sec):
```
Telemetry processMonitor heapUsedMB=2187, heapTotalMB=2234, cpuAverage=1.13, intervalMs=17911
Telemetry processMonitor heapUsedMB=2188, heapTotalMB=2234, cpuAverage=0.66, intervalMs=5005
Telemetry processMonitor heapUsedMB=2188, heapTotalMB=2234, cpuAverage=0, intervalMs=5005
Telemetry processMonitor heapUsedMB=71, heapTotalMB=75, cpuAverage=0.13, intervalMs=5002
```
After (total time to server request 18 sec):
```
Telemetry processMonitor heapUsedMB=109, heapTotalMB=144, cpuAverage=0.5, intervalMs=5001
Telemetry processMonitor heapUsedMB=109, heapTotalMB=144, cpuAverage=1.39, intervalMs=5002
Telemetry processMonitor heapUsedMB=94, heapTotalMB=131, cpuAverage=1.13, intervalMs=5000
Telemetry processMonitor heapUsedMB=94, heapTotalMB=131, cpuAverage=1.35, intervalMs=5001
```
Note in "Before" that heapTotalMB goes up to 2GB in the first case, and "intervalMs" of 17 seconds indicates that node was unresponsive for that long. In the second case, heapTotalMB stays low, and the main thread remains responsive the whole time.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3906
2023-06-01 12:06:48 -04:00
Paul Fitzpatrick
ae7d964bf2 (core) updates from grist-core 2023-05-08 09:34:11 -04:00
Jakub Serafin
37347a79c0 (core) Currency from grist column is persistent when exporting to excel
Summary:
- when grist table is exported, currency is check and introduced in cell format in the form of "[currency symbol] [value]" (for example: zł 10000, $ 5000) . It's not what some cultures should display currences, but it's close enought
- when no symbol is defined for the currency, currency 3 letters code is used instead
- when currency is unknown, we are falling back to "$"

Test Plan: - nbrowser test scenario added for that purpose, please check Currences.xlsx to see output format exported.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3886
2023-05-08 10:39:01 +02:00
Paul Fitzpatrick
1152976152
v1.1.0 2023-05-04 15:02:55 -04:00