Commit Graph

54 Commits

Author SHA1 Message Date
fflorent
9bacfc8287 Use chokidar to bundle css files 2024-04-29 14:54:36 -04:00
fflorent
0224f22705 Fix warning in webpack 2024-04-29 14:54:36 -04:00
fflorent
14061fd5ac Remove catw 2024-04-29 14:54:36 -04:00
Florent
4a9b6fea9d
Shutdown Doc worker when it is not considered as available in Redis #831 (#856)
* Shutdown Doc worker when it is not considered as available in Redis
* Use isAffirmative for GRIST_MANAGED_WORKERS
* Upgrade Sinon for the tests
* Run Smoke test with pages in English
* Add logic in /status endpoint
2024-04-04 10:25:42 -04:00
Thomas Steen Rasmussen
c40fa180c1
fix shebang in various bash scripts (#910) 2024-03-25 09:55:20 -04:00
Paul Fitzpatrick
95b734149e
make a /boot/GRIST_BOOT_KEY page for diagnosing configuration problems (#850)
This is a start at a page for diagnosing problems while setting up Grist. Starting to add some diagnostics based on feedback in github issues. We should make Grist installation easier! But when there is a problem it should be easier to diagnose than it is now, and this may help. The page is ugly and doesn't have many diagnostics yet, but we can iterate.

Visit `/boot` on a Grist server for tips on how to use this feature.
2024-03-04 16:22:47 -05:00
George Gevoian
c6fd79ac1f (core) Refactor forms implementation
Summary: WIP

Test Plan: Existing tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D4196
2024-02-22 08:44:25 -05:00
Dmitry S
11afc08f65 (core) Improve API Console and link from Document Settings.
Summary:
Changes to building and serving:
- Remove unpkg dependencies, add npm module for swagger-ui-dist instead.
- Move apiconsole JS logic into core/app/client/apiconsole.ts, and use TypeScript.
- Add symlinks to swagger in static/ and core/static/.
- Refactor loadScript, and add loadCssFile; use these to load swagger-ui resources.

Changes to console itself:
- Support docId, workspaceId, orgId URL parameters. When present, the matching
  value in dropdowns is moved to the front and marked as "(Current)".
- Fix the ordering of example values, particularly for workspaces.
- Remove unwanted example values.
- Hide confusing "Authorize" button.
- Hide API keys, and rely consistently on cookies for executing API calls.

Integration into Grist:
- Added a button to Document Settings, just under document ID in "API".
- The button opens a separate page, passing in org, workspace, and doc info for the current doc.

Test Plan: Only tested manually, no automated tests yet.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4173
2024-01-29 10:08:19 -05:00
George Gevoian
caf830db08 (core) Record Cards
Summary:
Adds a new Record Card view section to each non-summary table, which can be from opened from various parts of the Grist UI to view and edit records in a popup card view.

Work is still ongoing, so the feature is locked away behind a flag; follow-up work is planned to finish up the implementation and add end-to-end tests.

Test Plan: Python and server tests. Browser tests will be included in a follow-up.

Reviewers: jarek, paulfitz

Reviewed By: jarek

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4114
2023-11-19 20:12:37 -05:00
Jarosław Sadziński
3c219e05f6 (core) Removing the new menu flag
Summary: Enabling the `GRIST_NEW_COLUMN_MENU` flag by default and removing it.

Test Plan: Existing

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4098
2023-11-08 07:37:54 +01:00
Jakub Serafin
2521db4c55 (core) New Columns Menu
Summary:
A menu to be shown when new colum button is added. It's give access to various diffrent shortcuts, like adding new column, unhiding existing ones, fast adding lookup columns or trigger one (authoriship or timestamp). Design document can be found here: https://grist.quip.com/CTgxAQv9Ghjt/Add-Columns-more-easily
To turn on this menu flag GRIST_NEW_COLUMN_MENU to 1

Test Plan: UI tests suite under nbrowser/GridViewNewColumnMenu.ts

Reviewers: jarek, georgegevoian

Reviewed By: georgegevoian

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4074
2023-10-13 22:35:36 +02:00
Dmitry S
996674211d (core) Fix flyctl memory settings, since requested memory flag wasn't being used
Summary: After removal of 'scale' command in https://phab.getgrist.com/D3983, memory settings stopped applying properly to the actual machines that fly starts. This change restores them.

Test Plan: Tested by examining the memory allocated to a fly machine with and without this change.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4063
2023-10-02 11:24:17 -04:00
Jakub Serafin
cfc746d558 (core) fly.io version updated
Summary: fly.io made some breaking changes between so we need to bump up version from 1.16 to 1.18. Also, volumes cannot have name in more than 30 characters so i've introducent trimming mechanism for volume names

Test Plan: I've deployed preview with that settings and it went OK.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4033
2023-09-08 08:52:09 +02:00
Jakub Serafin
942fc96225 (core) custom widget appear as build-in widget
Summary: Added boilerplate code needed to create new wigets in "Add new" menu, that are wrapped around existing custom widgets. More details can be found here: https://grist.quip.com/larhAGRKyl6Z/Custom-widgets-in-Add-Widget-menu

Test Plan: nbowser tests added to verify if item in menu exits, if widget is rendered, and right side menu has widget selection and read access selection hided.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3994
2023-08-30 09:44:25 +02:00
Jakub Serafin
e387c6cf2c (core) update fly.yml to use 0.1.66 version of flyctl and to use V2 of flydev (so no scaling of Nomad)
Test Plan: manually run github action on github to check if applciation is deploying and destroing afterwards

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3983
2023-08-01 11:30:10 +02:00
Jakub Serafin
d894b60fd4 (core) deleting queue from single webhook
Summary: Using standard tost notification, message about webhook queue being overflown was added. message is permanent as long as queue is full. Message contains linkt to the webhook setings

Test Plan: two nbrowser test was added - one to check if message is show when queue is full, and second to check if message is dismiss when queue was cleaned.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3929
2023-07-18 11:46:10 +02:00
George Gevoian
35237a5835 (core) Add Support Grist page and nudge
Summary:
Adds a new Support Grist page (accessible only in grist-core), containing
options to opt in to telemetry and sponsor Grist Labs on GitHub.

A nudge is also shown in the doc menu, which can be collapsed or permanently
dismissed.

Test Plan: Browser and server tests.

Reviewers: paulfitz, dsagal

Reviewed By: paulfitz

Subscribers: jarek, dsagal

Differential Revision: https://phab.getgrist.com/D3926
2023-07-04 17:36:59 -04:00
Paul Fitzpatrick
70935a4fa4
skip building test harness in docker image (#551)
A small test harness bundle was recently added that is breaking the docker image build. It could be added to the docker image, but that would introduce a bunch of extraneous test file dependencies. So this tweaks the build to simply skip the test bundle if its primary source file is not found.

Also added some other test fixes along the way:
  * make a custom widget test more reliable
  * update a localization test now that `pt` exists
  * store more log info in artifact on error
2023-06-30 05:50:40 -04:00
Paul Fitzpatrick
fc16b4c8f6 (core) updates from grist-core 2023-06-27 03:04:42 -04:00
Paul Fitzpatrick
bcbf57d590 (core) bump mocha version to allow parallel tests; move more tests to core
Summary:
This uses a newer version of mocha in grist-core so that tests can be run in parallel. That allows more tests to be moved without slowing things down overall. Tests moved are venerable browser tests; only the ones that "just work" or worked without too much trouble to are moved, in order to keep the diff from growing too large. Will wrestle with more in follow up.

Parallelism is at the file level, rather than the individual test.

The newer version of mocha isn't needed for grist-saas repo; tests are parallelized in our internal CI by other means. I've chosen to allocate files to workers in a cruder way than our internal CI, based on initial characters rather than an automated process. The automated process would need some reworking to be compatible with mocha running in parallel mode.

Test Plan: this diff was tested first on grist-core, then ported to grist-saas so saas repo history will correctly track history of moved files.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3927
2023-06-27 02:55:34 -04:00
Dmitry S
331aa3152f (core) Fix flyctl cleanup; move/copy scripts to support preview deploys in grist-core
Summary:
- Fly flyctl cleanup which has been failing; deleting volumes before app no longer seems needed
- Move fly-deploy and add workflow scripts to make fly preview work in grist-core too

Test Plan: Tested that previews still work in this repo; and tested with a copy in grist-core.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3928
2023-06-22 12:13:21 -04:00
Florent
1e7cf9001e
Remove unused GRIST_EXT env variable #531 (#532)
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-06-13 11:13:07 -04:00
Paul Fitzpatrick
7be0ee289d
support other SQLite wrappers, and various hooks needed by grist-static (#516) 2023-05-23 15:17:28 -04:00
Paul Fitzpatrick
f7f76fb5e7
A set of tweaks to simplify electron packaging (#421)
* Replace `ormconfig.js` with a newer mechanism of configuring
    TypeORM that can be included in the source code properly.
    The path to `ormconfig.js` has always been awkward to handle,
    and eliminating the file makes building different Grist setups
    a bit simpler.
  * Remove `electron` package. It is barely used, just for some old
    remnants of an older attempt at electron packaging. It was used
    for two types, which I left at `any` for now. More code pruning is
    no doubt possible here, but I'd rather do it when Electron packaging
    has solidified.
  * Add a hook for replacing the login system, and for adding some
    extra middleware the login system may need.
  * Add support for some more possible locations of Python, which
    arise when a standalone version of it is included in the Electron
    package. This isn't very general purpose, just configurations
    that I found useful.
  * Support using grist-core within a yarn workspace - the only tweak
    needed was webpack related.
  * Allow an external ID to be optionally associated with documents.
2023-02-13 15:52:17 -05:00
Paul Fitzpatrick
9d93c2e6c9 stub -> stubs 2023-01-17 17:09:59 -05:00
Paul Fitzpatrick
6296423248 update paths for grist-core layout 2023-01-17 16:16:36 -05:00
Paul Fitzpatrick
e5e44c786a add a script for copying schema information from python to typescript
There was no script for updating typescript schema information after
a python-based document migration. Moving one in here, along with its
test. Tweaked the code slightly to work with grist-core's directory
structure. Also fixed a formatting error in mocha calls that was resulting
in some root tests not running.
2023-01-17 15:54:41 -05:00
Paul Fitzpatrick
b8e964564c report unrecognized keys; be careful to not clobber English translation of single/plural forms 2023-01-12 12:54:14 -05:00
Paul Fitzpatrick
0d8b7e80f6 tweak translation collection to be weblate-friendly
This tweaks the script for collecting translation keys so that it
changes the existing file minimally, and matches the formatting
output by weblate.

I'm not sure what is the best way to handle keys that are no longer
needed. I think it may be best to delete these within weblate, but
will need to experiment to see (I'm a weblate newbie). I think I
saw keys reappear from weblate if not deleted there.
2023-01-11 12:24:01 -05:00
Paul Fitzpatrick
e6692c2793 (core) add a checkbox for owner "boss mode"
Summary:
Implement a checkbox that grants owners full access to tables by default, when creating new table/column rules.
 * Checkbox appears above default rules.
 * When set, a rule giving owners full access will be inserted in any new rule set started for tables or columns.
 * The checkbox can be expanded to allow customization of the rules.

https://gristlabs.getgrist.com/doc/check-ins/p/3#a1.s7.r2251.c19

Test Plan: added tests

Reviewers: jarek

Reviewed By: jarek

Subscribers: anaisconce

Differential Revision: https://phab.getgrist.com/D3756
2023-01-09 13:20:23 -05:00
Louis Delbosc
c18c6cb264 Clean generating script and english translation values 2023-01-03 16:01:45 +01:00
Louis Delbosc
4165c35dd3 Add translation keys generator script 2023-01-03 16:01:45 +01:00
Paul Fitzpatrick
6dd2068218 (core) show package.json version when hovering on Grist icon in grist-core
Summary:
This makes the version shown when hovering on the Grist icon equal
the version set in package.json, for a grist-core build. Previously
the number shown was a hard-coded placeholder.

The Grist SaaS build has some build machinery dealing with the
version number that should be unaffected by this change for now.

Test Plan: tested manually with build_core.sh

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3659
2022-10-12 16:02:01 -04:00
Jarosław Sadziński
2438a63255 (core) Moving widget tests to core
Summary:
- Custom widget tests are now in grist-core
- Adding buildtools for grist-plugin-api.js

Test Plan: Existing tests

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3617
2022-09-06 17:17:14 +02:00
George Gevoian
ed37401b2c (core) Add basic activation page to grist-ee
Summary:
Adds an activation page to grist-ee that currently shows activation status.

Follow-up diffs will introduce additional functionality, such as the ability to
enter activation keys directly from the activation page.

Test Plan: No grist-ee tests (yet).

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: dsagal, paulfitz

Differential Revision: https://phab.getgrist.com/D3582
2022-08-23 10:30:52 -07:00
Dmitry S
51ff72c15e (core) Faster builds all around.
Summary:
Building:
- Builds no longer wait for tsc for either client, server, or test targets. All use esbuild which is very fast.
- Build still runs tsc, but only to report errors. This may be turned off with `SKIP_TSC=1` env var.
- Grist-core continues to build using tsc.
- Esbuild requires ES6 module semantics. Typescript's esModuleInterop is turned
  on, so that tsc accepts and enforces correct usage.
- Client-side code is watched and bundled by webpack as before (using esbuild-loader)

Code changes:
- Imports must now follow ES6 semantics: `import * as X from ...` produces a
  module object; to import functions or class instances, use `import X from ...`.
- Everything is now built with isolatedModules flag. Some exports were updated for it.

Packages:
- Upgraded browserify dependency, and related packages (used for the distribution-building step).
- Building the distribution now uses esbuild's minification. babel-minify is no longer used.

Test Plan: Should have no behavior changes, existing tests should pass, and docker image should build too.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3506
2022-07-04 10:42:40 -04:00
Dmitry S
dd2eadc86e (core) Speed up and upgrade build.
Summary:
- Upgrades to build-related packages:
  - Upgrade typescript, related libraries and typings.
  - Upgrade webpack, eslint; add tsc-watch, node-dev, eslint_d.

- Build organization changes:
  - Build webpack from original typescript, transpiling only; with errors still
    reported by a background tsc watching process.

- Typescript-related changes:
  - Reduce imports of AWS dependencies (very noticeable speedup)
  - Avoid auto-loading global @types
  - Client code is now built with isolatedModules flag (for safe transpilation)
  - Use allowJs to avoid copying JS files manually.

- Linting changes
  - Enhance Arcanist ESLintLinter to run before/after commands, and set up to use eslint_d
  - Update eslint config, and include .eslintignore to avoid linting generated files.
  - Include a bunch of eslint-prompted and eslint-generated fixes
  - Add no-unused-expression rule to eslint, and fix a few warnings about it

- Other items:
  - Refactor cssInput to avoid circular dependency
  - Remove a bit of unused code, libraries, dependencies

Test Plan: No behavior changes, all existing tests pass. There are 30 tests fewer reported because `test_gpath.py` was removed (it's been unused for years)

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3498
2022-06-27 16:10:10 -04:00
Alex Hall
9fffb491f9 (core) External requests
Summary:
Adds a Python function `REQUEST` which makes an HTTP GET request. Behind the scenes it:

- Raises a special exception to stop trying to evaluate the current cell and just keep the existing value.
- Notes the request arguments which will be returned by `apply_user_actions`.
- Makes the actual request in NodeJS, which sends back the raw response data in a new action `RespondToRequests` which reevaluates the cell(s) that made the request.
- Wraps the response data in a class which mimics the `Response` class of the `requests` library.

In certain cases, this asynchronous flow doesn't work and the sandbox will instead synchronously call an exported JS method:

- When reevaluating a single cell to get a formula error, the request is made synchronously.
- When a formula makes multiple requests, the earlier responses are retrieved synchronously from files which store responses as long as needed to complete evaluating formulas. See https://grist.slack.com/archives/CL1LQ8AT0/p1653399747810139

Test Plan: Added Python and nbrowser tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: paulfitz, dsagal

Differential Revision: https://phab.getgrist.com/D3429
2022-06-17 21:53:20 +02:00
Paul Fitzpatrick
e6983e9209 (core) add machinery for self-managed flavor of Grist
Summary:
Currently, we have two ways that we deliver Grist. One is grist-core,
which has simple defaults and is relatively easy for third parties to
deploy. The second is our internal build for our SaaS, which is the
opposite. For self-managed Grist, a planned paid on-premise version
of Grist, I adopt the following approach:

 * Use the `grist-core` build mechanism, extending it to accept an
   overlay of extra code if present.
 * Extra code is supplied in a self-contained `ext` directory, with
   an `ext/app` directory that is of same structure as core `app`
   and `stubs/app`.
 * The `ext` directory also contains information about extra
   node dependencies needed beyond that of `grist-core`.
 * The `ext` directory is contained within our monorepo rather than
   `grist-core` since it may contain material not under the Apache
   license.

Docker builds are achieved in our monorepo by using the `--build-context`
functionality to add in `ext` during the regular `grist-core` build:

```
docker buildx build --load -t gristlabs/grist-ee --build-context=ext=../ext .
```

Incremental builds in our monorepo are achieved with the `build_core.sh` helper,
like:

```
buildtools/build_core.sh /tmp/self-managed
cd /tmp/self-managed
yarn start
```

The initial `ext` directory contains material for snapshotting to S3.
If you build the docker image as above, and have S3 access, you can
do something like:

```
docker run -p 8484:8484 --env GRIST_SESSION_SECRET=a-secret \
  --env GRIST_DOCS_S3_BUCKET=grist-docs-test \
  --env GRIST_DOCS_S3_PREFIX=self-managed \
  -v $HOME/.aws:/root/.aws -it gristlabs/grist-ee
```

This will start a version of Grist that is like `grist-core` but with
S3 snapshots enabled. To release this code to `grist-core`, it would
just need to move from `ext/app` to `app` within core.

I tried a lot of ways of organizing self-managed Grist, and this was
what made me happiest. There are a lot of trade-offs, but here is what
I was looking for:

 * Only OSS-code in grist-core. Adding mixed-license material there
   feels unfair to people already working with the repo. That said,
   a possible future is to move away from our private monorepo to
   a public mixed-licence repo, which could have the same relationship
   with grist-core as the monorepo has.
 * Minimal differences between self-managed builds and one of our
   existing builds, ideally hewing as close to grist-core as possible
   for ease of documentation, debugging, and maintenance.
 * Ideally, docker builds without copying files around (the new
   `--build-context` functionality made that possible).
 * Compatibility with monorepo build.

Expressing dependencies of the extra code in `ext` proved tricky to
do in a clean way. Yarn/npm fought me every step of the way - everything
related to optional dependencies was unsatisfactory in some respect.
Yarn2 is flexible but smells like it might be overreach. In the end,
organizing to install non-core dependencies one directory up from the
main build was a good simple trick that saved my bacon.

This diff gets us to the point of building `grist-ee` images conveniently,
but there isn't a public repo people can go look at to see its source. This
could be generated by taking `grist-core`, adding the `ext` directory
to it, and pushing to a distinct repository. I'm not in a hurry to do that,
since a PR to that repo would be hard to sync with our monorepo and
`grist-core`. Also, we don't have any licensing text ready for the `ext`
directory. So leaving that for future work.

Test Plan: manual

Reviewers: georgegevoian, alexmojaki

Reviewed By: georgegevoian, alexmojaki

Differential Revision: https://phab.getgrist.com/D3415
2022-05-12 12:39:52 -04:00
Thomas Karolski
953ac7c689 Use /usr/bin/env instead of /bin/bash 2022-03-08 19:24:36 +00:00
George Gevoian
ba6ecc5e9e (core) Move user profile to new page and begin MFA work
Summary:
The user profile dialog is now a separate page, in preparation
for upcoming work to enable MFA. This commit also contains
some MFA changes, but the UI is currently disabled and the
implementation is limited to software tokens (TOTP) only.

Test Plan:
Updated browser tests for new profile page. Tests for MFAConfig
and CognitoClient will be added in a later diff, once the UI is enabled.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3199
2022-01-13 21:21:49 -08:00
Paul Fitzpatrick
5cdc7b2ea4 (core) freshen core README; support python3 in grist-core docker image
Summary:
This updates the grist-core README to list specific features of Grist,
to make it easier for a casual visitor to get a sense of its scope. Adds links
to some new resources (reviews, templates, grist v airtable post) that could
also help. Adds python3 to docker image so that templates work without fuss.

Test Plan: existing tests should pass

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal, anaisconce

Differential Revision: https://phab.getgrist.com/D3204
2022-01-08 18:27:20 -05:00
Paul Fitzpatrick
975eed8564 (core) fix browser check and favicon in grist-core
Summary:
A check for old browsers and a Grist favicon were not available in
grist-core, leaving harmless but distracting errors in logs. This
adds them.

Test Plan: checked manually

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3207
2022-01-05 18:15:05 -05:00
Jarosław Sadziński
048c8ee165 (core) Extending default locale list
Summary: Adding more locale codes to support more countries in document settings

Test Plan: existing tests

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3018
2021-09-24 15:10:13 +02:00
Paul Fitzpatrick
f9630b3aa4 (core) clean up a collection of small problems affecting grist-core
Summary:
 * Remove adjustSession hack, interfering with loading docs under saml.
 * Allow the anonymous user to receive an empty list of workspaces for
   the merged org.
 * Behave better on first page load when org is in path - this used to
   fail because of lack of cookie.  This is very visible in grist-core,
   as a failure to load localhost:8484 on first visit.
 * Mark cookie explicitly as SameSite=Lax to remove a warning in firefox.
 * Make errorPages available in grist-core.

This changes the default behavior of grist-core to now start off in
anonymous mode, with an explicit sign-in step available.  If SAML is not configured,
the sign-in operation will unconditionally sign the user in as a default
user, without any password check or other security.  The user email is
taken from GRIST_DEFAULT_EMAIL if set.  This is a significant change, but
makes anonymous mode available in grist-core (which is convenient
for testing) and makes behavior with and without SAML much more consistent.

Test Plan: updated test; manual (time to start adding grist-core tests though!)

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2980
2021-08-17 21:44:50 -04:00
Paul Fitzpatrick
bb8cb2593d (core) support python3 in grist-core, and running engine via docker and/or gvisor
Summary:
 * Moves essential plugins to grist-core, so that basic imports (e.g. csv) work.
 * Adds support for a `GRIST_SANDBOX_FLAVOR` flag that can systematically override how the data engine is run.
   - `GRIST_SANDBOX_FLAVOR=pynbox` is "classic" nacl-based sandbox.
   - `GRIST_SANDBOX_FLAVOR=docker` runs engines in individual docker containers. It requires an image specified in `sandbox/docker` (alternative images can be named with `GRIST_SANDBOX` flag - need to contain python and engine requirements). It is a simple reference implementation for sandboxing.
   - `GRIST_SANDBOX_FLAVOR=unsandboxed` runs whatever local version of python is specified by a `GRIST_SANDBOX` flag directly, with no sandboxing. Engine requirements must be installed, so an absolute path to a python executable in a virtualenv is easiest to manage.
   - `GRIST_SANDBOX_FLAVOR=gvisor` runs the data engine via gvisor's runsc. Experimental, with implementation not included in grist-core. Since gvisor runs on Linux only, this flavor supports wrapping the sandboxes in a single shared docker container.
 * Tweaks some recent express query parameter code to work in grist-core, which has a slightly different version of express (smoke test doesn't catch this since in Jenkins core is built within a workspace that has node_modules, and wires get crossed - in a dev environment the problem on master can be seen by doing `buildtools/build_core.sh /tmp/any_path_outside_grist`).

The new sandbox options do not have tests yet, nor does this they change the behavior of grist servers today. They are there to clean up and consolidate a collection of patches I've been using that were getting cumbersome, and make it easier to run experiments.

I haven't looked closely at imports beyond core.

Test Plan: tested manually against regular grist and grist-core, including imports

Reviewers: alexmojaki, dsagal

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D2942
2021-07-28 09:02:32 -04:00
Dmitry S
526b0ad33e (core) Configure more comprehensive eslint rules for Typescript
Summary:
- Update rules to be more like we've had with tslint
- Switch tsserver plugin to eslint (tsserver makes for a much faster way to lint in editors)
- Apply suggested auto-fixes
- Fix all lint errors and warnings in core/, app/, test/

Test Plan: Some behavior may change subtly (e.g. added missing awaits), relying on existing tests to catch problems.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2785
2021-04-26 18:54:55 -04:00
Paul Fitzpatrick
45141ed438 (core) add CI github action for grist-core
Summary:
Activate CI for grist-core using github actions.  Can be improved
with caching but starting simple.

Test Plan: tested in a fork of grist-core

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2771
2021-04-12 17:17:17 -04:00
Paul Fitzpatrick
1654a2681f (core) move client code to core
Summary:
This moves all client code to core, and makes minimal fix-ups to
get grist and grist-core to compile correctly.  The client works
in core, but I'm leaving clean-up around the build and bundles to
follow-up.

Test Plan: existing tests pass; server-dev bundle looks sane

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2627
2020-10-02 13:24:21 -04:00
Paul Fitzpatrick
b82eec714a (core) move data engine code to core
Summary:
this moves sandbox/grist to core, and adds a requirements.txt
file for reconstructing the content of sandbox/thirdparty.

Test Plan:
existing tests pass.
Tested core functionality manually.  Tested docker build manually.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2563
2020-07-29 08:57:25 -04:00