Commit Graph

28 Commits

Author SHA1 Message Date
Florent
786ba6b31e
Move HomeDBManager to gen-server/lib/homedb (#1076) 2024-07-05 16:02:39 +02:00
Florent
5e3cd94177
Introduce APP_HOME_INTERNAL_URL and fix duplicate docs (#915)
Context:

On self-hosted instances, some places in the code rely on the fact that we resolves public domains while being behind reverse proxies. This leads to cases where features are not available, such as the "Duplicate document" one.

Bugs that are solved - n self-hosted instances:

Impossible to open templates and tutorials right after having converted them;
Impossible to submit forms since version 1.1.13;
Impossible to restore a previous version of a document (snapshot);
Impossible to copy a document;

Solution:

Introduce the APP_HOME_INTERNAL_URL env variable, which is quite the same as APP_DOC_INTERNAL_URL except that it may point to any home worker;
Make /api/worker/:assignmentId([^/]+)/?* return not only the doc worker public url but also the internal one, and adapt the call points like fetchDocs;
Ensure that the home and doc worker internal urls are trusted by trustOrigin;

---------

Co-authored-by: jordigh <jordigh@octave.org>
2024-05-14 12:58:41 -04:00
George Gevoian
c6fd79ac1f (core) Refactor forms implementation
Summary: WIP

Test Plan: Existing tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D4196
2024-02-22 08:44:25 -05:00
George Gevoian
b8f32d1784 (core) Add additional telemetry events
Summary: The new events capture usage of forms, widgets, access rules, and onboarding tours and tips.

Test Plan: Manual.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D4189
2024-02-13 13:09:16 -05:00
George Gevoian
cb298e63d4 (core) Fix bug forcing login on some form URLs
Summary:
Login (and other) middleware was included in the public form URL by mistake,
forcing logins on forms hosted on non-personal sites.

Test Plan: Browser test.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D4181
2024-02-01 12:10:52 -05:00
George Gevoian
6cb8614017 (core) Polish forms
Summary:
  - Updates styling of form submitted page.
  - Tweaks styling of checkboxes, labels, and questions on form page.
  - Adds new form 404 page.
  - Adds checkbox to not show warning again when publishing or un-publishing a form.
  - Excludes formula, hidden, and attachment columns in submitted form data.
  - Adds placeholder text to form configuration inputs.
  - Improves dark mode styling in Form widget.
  - Updates default title and description of new forms.
  - Updates styling of Form widget buttons.
  - Fixes form success text input handling.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4170
2024-01-24 02:18:03 -08:00
George Gevoian
e12471347b (core) Form Publishing
Summary:
Adds initial implementation of form publishing, built upon WYSIWYS shares.

A simple UI for publishing and unpublishing forms is included.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: paulfitz, jarek

Differential Revision: https://phab.getgrist.com/D4154
2024-01-12 11:58:12 -08:00
Paul Fitzpatrick
2a206dfcf8 (core) add initial support for special shares
Summary:
This gives a mechanism for controlling access control within a document that is distinct from (though implemented with the same machinery as) granular access rules.

It was hard to find a good way to insert this that didn't dissolve in a soup of complications, so here's what I went with:
 * When reading rules, if there are shares, extra rules are added.
 * If there are shares, all rules are made conditional on a "ShareRef" user property.
 * "ShareRef" is null when a doc is accessed in normal way, and the row id of a share when accessed via a share.

There's no UI for controlling shares (George is working on it for forms), but you can do it by editing a `_grist_Shares` table in a document. Suppose you make a fresh document with a single page/table/widget, then to create an empty share you can do:

```
gristDocPageModel.gristDoc.get().docData.sendAction(['AddRecord', '_grist_Shares', null, {linkId: 'xyz', options: '{"publish": true}'}])
```

If you look at the home db now there should be something in the `shares` table:

```
$ sqlite3 -table landing.db "select * from shares"
+----+------------------------+------------------------+--------------+---------+
| id |          key           |         doc_id         |   link_id    | options |
+----+------------------------+------------------------+--------------+---------+
| 1  | gSL4g38PsyautLHnjmXh2K | 4qYuace1xP2CTcPunFdtan | xyz | ...      |
+----+------------------------+------------------------+--------------+---------+
```

If you take the key from that (gSL4g38PsyautLHnjmXh2K in this case) and replace the document's urlId in its URL with `s.<key>` (in this case `s.gSL4g38PsyautLHnjmXh2K` then you can use the regular document landing page (it will be quite blank initially) or API endpoint via the share.

E.g. for me `http://localhost:8080/o/docs/s0gSL4g38PsyautLHnjmXh2K/share-inter-3` accesses the doc.

To actually share some material - useful commands:

```
gristDocPageModel.gristDoc.get().docData.getMetaTable('_grist_Views_section').getRecords()
gristDocPageModel.gristDoc.get().docData.sendAction(['UpdateRecord', '_grist_Views_section', 1, {shareOptions: '{"publish": true, "form": true}'}])
gristDocPageModel.gristDoc.get().docData.getMetaTable('_grist_Pages').getRecords()
gristDocPageModel.gristDoc.get().docData.sendAction(['UpdateRecord', '_grist_Pages', 1, {shareRef: 1}])
```

For a share to be effective, at least one page needs to have its shareRef set to the rowId of the share, and at least one widget on one of those pages needs to have its shareOptions set to {"publish": "true", "form": "true"} (meaning turn on sharing, and include form sharing), and the share itself needs {"publish": true} on its options.

I think special shares are kind of incompatible with public sharing, since by their nature (allowing access to all endpoints) they easily expose the docId, and changing that would be hard.

Test Plan: tests added

Reviewers: dsagal, georgegevoian

Reviewed By: dsagal, georgegevoian

Subscribers: jarek, dsagal

Differential Revision: https://phab.getgrist.com/D4144
2024-01-04 05:57:38 -05:00
Alex Hall
a2bd753649 (core) API console
Summary: Adds a custom interactive Swagger API console at `/apiconsole`. For now, this isn't visibly linked anywhere.

Test Plan: Manual, this is still an experimental and private feature. The idea is to merge this soon so that we have a chance to try it out in production.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4151
2023-12-27 22:02:30 +02:00
Paul Fitzpatrick
bcb9740d89 (core) update tests and endpoints after bare /urlId redirect added
Summary:
 * Some tests needed updating because fake document ids in tests were changed to be valid urlIds (the existing ones were too short).
 * urlId capture is tweaked to not allow hyphens, so some long login-related paths don't get confused with documents.

Test Plan: tests should pass again

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D4134
2023-12-04 11:14:10 -05:00
Florent
cf0cbb404e
Allow URLs with only a docID #768 (#771)
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-11-29 15:13:29 -05:00
George Gevoian
7a85aaa7a1 (core) Add new telemetry events
Summary: Adds a handful of new telemetry events, and makes a few tweaks to allow for better organization of telemetry.

Test Plan: Manual.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D4100
2023-11-01 10:49:33 -04:00
George Gevoian
3dadf93c98 (core) Add support for auto-copying docs on signup
Summary:
The new "copyDoc" query parameter on the login page sets a short-lived cookie, which is
then read when welcoming a new user to copy that document to their Home workspace, and
redirect to it. Currently, only templates and bare forks set this parameter.

A new API endpoint for copying a document to a workspace was also added.

Test Plan: Browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3992
2023-09-06 15:12:08 -04:00
Paul Fitzpatrick
61f954ff05
move getTemplateOrg method; enable template org in docker tests (#602)
* move getTemplateOrg method; enable template org in docker tests

This moves the `getTemplateOrg` method to a neutral venue for the
convenience of `grist-static`, otherwise a lot of awkward dependencies
get pulled in needlessly in new parts of the app.

This also fixes docker tests using the template org.
2023-07-31 16:10:59 -04:00
George Gevoian
a77170c4bd (core) Tweak navbar, breadcrumbs, and sign-in buttons
Summary:
The changes are intended to smooth over some sharp edges when a signed-out user
is using Grist (particularly while on the templates site).

Test Plan: Browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3957
2023-07-26 22:26:55 -07:00
George Gevoian
35237a5835 (core) Add Support Grist page and nudge
Summary:
Adds a new Support Grist page (accessible only in grist-core), containing
options to opt in to telemetry and sponsor Grist Labs on GitHub.

A nudge is also shown in the doc menu, which can be collapsed or permanently
dismissed.

Test Plan: Browser and server tests.

Reviewers: paulfitz, dsagal

Reviewed By: paulfitz

Subscribers: jarek, dsagal

Differential Revision: https://phab.getgrist.com/D3926
2023-07-04 17:36:59 -04:00
George Gevoian
10f5f0cb37 (core) Add optional telemetry to grist-core
Summary:
Adds support for optional telemetry to grist-core.

A new environment variable, GRIST_TELEMETRY_LEVEL, controls the level of telemetry collected.

Test Plan: Server and unit tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: dsagal, anaisconce

Differential Revision: https://phab.getgrist.com/D3880
2023-06-07 12:00:51 -04:00
George Gevoian
1e873b4203 (core) Tweak telemetry
Summary: Adjusts the level of telemetry collected from Grist SaaS.

Test Plan: Tested manually.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3899
2023-05-19 13:06:15 -04:00
George Gevoian
a19ba0813a (core) Add telemetry
Test Plan: Server tests.

Reviewers: jarek

Differential Revision: https://phab.getgrist.com/D3818
2023-04-06 12:34:54 -04:00
George Gevoian
be8e13df64 (core) Add initial tutorials implementation
Summary:
Documents can now be flagged as tutorials, which causes them to display
Markdown-formatted slides from a special GristDocTutorial table. Tutorial
documents are forked on open, and remember the last slide a user was on.
They can be restarted too, which prepares a new fork of the tutorial.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3813
2023-03-22 10:09:02 -04:00
Paul Fitzpatrick
177b9d83d9 (core) add a log message on proxying failures
Summary:
When a home server fails to fetch from a doc worker, errors are
passed along to clients but we don't get to see them. This fixes
that omission.

Test Plan:
tested manually, by inserting some code to delay
serving particular test documents.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3601
2022-08-24 09:16:19 -04:00
Dmitry S
51ff72c15e (core) Faster builds all around.
Summary:
Building:
- Builds no longer wait for tsc for either client, server, or test targets. All use esbuild which is very fast.
- Build still runs tsc, but only to report errors. This may be turned off with `SKIP_TSC=1` env var.
- Grist-core continues to build using tsc.
- Esbuild requires ES6 module semantics. Typescript's esModuleInterop is turned
  on, so that tsc accepts and enforces correct usage.
- Client-side code is watched and bundled by webpack as before (using esbuild-loader)

Code changes:
- Imports must now follow ES6 semantics: `import * as X from ...` produces a
  module object; to import functions or class instances, use `import X from ...`.
- Everything is now built with isolatedModules flag. Some exports were updated for it.

Packages:
- Upgraded browserify dependency, and related packages (used for the distribution-building step).
- Building the distribution now uses esbuild's minification. babel-minify is no longer used.

Test Plan: Should have no behavior changes, existing tests should pass, and docker image should build too.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3506
2022-07-04 10:42:40 -04:00
Paul Fitzpatrick
2563fb745a (core) make Grist easier to run with a single server
Summary:
This makes many small changes so that Grist is less fussy to run as a single instance behind a reverse proxy. Some users had difficulty with the self-connections Grist would make, due to internal network setup, and since these are unnecessary in any case in this scenario, they are now optimized away. Likewise some users had difficulties related to doc worker urls, which are now also optimized away. With these changes, users should be able to get a lot further on first try, at least far enough to open and edit documents.

The `GRIST_SINGLE_ORG` setting was proving a bit confusing, since it appeared to only work when set to `docs`. This diff
adds a check for whether the specified org exists, and if not, it creates it. This still depends on having a user email to make as the owner of the team, so there could be remaining difficulties there.

Test Plan: tested manually with nginx

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3299
2022-03-05 13:30:45 -05:00
Jarosław Sadziński
4ca47878ca (core) Adding import from google drive to the home screen
Summary: Importing from google drive from home screen (also for anonymous users)

Test Plan: Browser tests

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2943
2021-08-05 20:46:11 +02:00
Dmitry S
526b0ad33e (core) Configure more comprehensive eslint rules for Typescript
Summary:
- Update rules to be more like we've had with tslint
- Switch tsserver plugin to eslint (tsserver makes for a much faster way to lint in editors)
- Apply suggested auto-fixes
- Fix all lint errors and warnings in core/, app/, test/

Test Plan: Some behavior may change subtly (e.g. added missing awaits), relying on existing tests to catch problems.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2785
2021-04-26 18:54:55 -04:00
Paul Fitzpatrick
45d2d5f897 (core) back-end support for tables that are accessible only by owners
Summary:
This makes it possible to serve a table or tables only to owners.

 * The _grist_ACLResources table is abused (temporarily) such that rows of the form `{colId: '~o', tableId}` are interpreted as meaning that `tableId` is private to owners.
 * Many websocket and api endpoints are updated to preserve the privacy of these tables.
 * In a document where some tables are private, a lot of capabilities are turned off for non-owners to avoid leaking info indirectly.
 * The client is tweaked minimally, to show '-' where a page with some private material would otherwise go.

No attempt is made to protect data from private tables pulled into non-private tables via formulas.

There are some known leaks remaining:
 * Changes to the schema of private tables are still broadcast to all clients (fixable).
 * Non-owner may be able to access snapshots or make forks or use other corners of API (fixable).
 * Changing name of table makes it public, since tableId in ACLResource is not updated (fixable).

Security will require some work, the attack surface is large.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2604
2020-09-14 18:05:27 -04:00
Paul Fitzpatrick
4a545c9f2a (core) make sharing with everyone@ on documents effective
Summary:
Sharing a document with everyone@ was effective at the api level,
but had two flaws in the web client:

 * A logged in user with no access at the org level could not access
   a publically shared doc within that org.
 * Likewise, for the anonymous user (but for a different reason).

This diff tweaks the web client to permit accessing a doc when
org information is unavailable.

It also changes how redirects happen for the anonymous user when
accessing a doc.  They now only happen once it has been confirmed
that the user does not have access to the doc.

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2591
2020-08-19 19:42:26 -04:00
Paul Fitzpatrick
5ef889addd (core) move home server into core
Summary: This moves enough server material into core to run a home server.  The data engine is not yet incorporated (though in manual testing it works when ported).

Test Plan: existing tests pass

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2552
2020-07-21 20:39:10 -04:00