Without this optimization, we fetched loads of entries from the database, which led to database and nodejs overloads.
We could go further, this is a modest patch towards better performance.
We use two queries: one fetches the workspaces, the second the organization that the workspace belongs to.
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary:
Creator panel was showing a header with tabs on Raw Data page. This
was broken by forms feature lately.
Test Plan: Added
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4179
Summary:
Improvements
- Widget and column descriptions are now copied when duplicating a table.
- A Grist Plugin API command to open a Record Card is now available.
- New Card widgets set initial settings based on those used by their table's
Record Card.
Fixes
- Opening a reference in a Record Card from a Raw Data popup now opens
the correct reference.
Test Plan: Browser and python tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4164
* run.sh: Replace pid with nodejs's one using exec
This notably ensures that "docker stop" sends a TERM signal to the node
process which can handle it gracefully and avoid document corruption.
* Use exec form for CMD in Dockerfile
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary:
Changes to building and serving:
- Remove unpkg dependencies, add npm module for swagger-ui-dist instead.
- Move apiconsole JS logic into core/app/client/apiconsole.ts, and use TypeScript.
- Add symlinks to swagger in static/ and core/static/.
- Refactor loadScript, and add loadCssFile; use these to load swagger-ui resources.
Changes to console itself:
- Support docId, workspaceId, orgId URL parameters. When present, the matching
value in dropdowns is moved to the front and marked as "(Current)".
- Fix the ordering of example values, particularly for workspaces.
- Remove unwanted example values.
- Hide confusing "Authorize" button.
- Hide API keys, and rely consistently on cookies for executing API calls.
Integration into Grist:
- Added a button to Document Settings, just under document ID in "API".
- The button opens a separate page, passing in org, workspace, and doc info for the current doc.
Test Plan: Only tested manually, no automated tests yet.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4173
Summary:
This offers better handling for accented and non-English characters,
e.g. "Événements" becomes "Evenements" (rather than "vnements") and
"таблиця" becomes "tablicya" rather than an empty string.
Test Plan:
Added a test case. Existing documents will auto-redirect to
newly-generated names. In cases where it's different, that's probably for the
best.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4176
Summary:
Takes some numbers embedded in strings and moves them to the context, so
they can be aggregated and trends inspected more easily.
Test Plan: tested manually by looking at logs
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4175
Summary: The "Build your own form" link now links to the Grist homepage.
Test Plan: Manual.
Reviewers: jarek
Reviewed By: jarek
Subscribers: jarek
Differential Revision: https://phab.getgrist.com/D4174
Summary:
- Adding little green asterisk at the end of field title.
- Fixing bug on columns component. Adding paragraph as a column and then selecting it was throwing error in the RightPanel
- Fixing boolean column bug in the editor
- Adding (--Choose--) placeholder for dropdowns
- Fixing columns logic: Dragging and dropping columns can unexpectedly add more columns.
- Added favicon and default page title
- Added svg to sync file for electron.
Test Plan: Updated
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4172
Summary:
- Updates styling of form submitted page.
- Tweaks styling of checkboxes, labels, and questions on form page.
- Adds new form 404 page.
- Adds checkbox to not show warning again when publishing or un-publishing a form.
- Excludes formula, hidden, and attachment columns in submitted form data.
- Adds placeholder text to form configuration inputs.
- Improves dark mode styling in Form widget.
- Updates default title and description of new forms.
- Updates styling of Form widget buttons.
- Fixes form success text input handling.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4170
Summary:
Enhances sign-up telemetry with login and verification method metadata, and
adds UTM parameters to SendGrid invite email links and Grist document links.
Test Plan: Server and manual.
Reviewers: dsagal
Reviewed By: dsagal
Subscribers: dsagal
Differential Revision: https://phab.getgrist.com/D4169
Summary:
This moves the `formUrl` logic to `encodeUrl`, which is more
aware of how the URL is constructed than UserAPI. UserAPI can
only reliably construct API URLs.
Test Plan: extended tests
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4171
Summary:
- Open all links in a new tab
- Excluding not filled columns (to fix trigger formulas)
- Fixed Ref/RefList submission
- Removing redundant type definitions for Box
- Adding header menu item
- Default empty values in select control
Test Plan: Updated
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4166
* Fix server crash when client passes malformed JSON
* Take remarks into account
---------
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
Summary:
The first time a worker opens a document, it will now check if it has any shares the home db needs to be aware of. If so, they will be added. This is important for documents uploaded/copied/forked/replaced, so that their shares work out of the box.
In future, may want some UI to give user control of whether shares are activated after upload/copy/fork/replace.
It seems tricky currently to know if a document is being opened for the first time. As a proxy, I check whether usage information has been calculated and saved to the db, since I can determine that without adding another db query. It is safe to synchronize shares more than necessary.
This leaves two gaps:
* If a document is created/uploaded/copied/forked/replaced and no attempt is made to access it prior to using a share, then that share won't actually be available. Not a problem currently I think, since how would a user have determined the share key. But in future it would be good to also do a sync after creation/upload/copy/fork/replacement/...
* On document replacement, usage info is reset but not absolutely immediately. So in principle shares could fail to be created on first load of the replacement. Usage info reset could be tweaked to give a guarantee here, but also fixing the first point would resolve this second point too.
Test Plan: copy test added
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4165
Summary:
This corresponding change in HelpCenter will report this event:
https://github.com/gristlabs/grist-help/pull/303
Test Plan: Tested locally that the telemetry event is received by the server and logged.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4168
Summary:
The "Create a form" button in section menu wasn't hidden
behind a feature flag.
Test Plan: Manual
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4167
Summary:
Forms improvements and following new design
- New headers
- New UI
- New right panel options
Test Plan: Tests updated
Reviewers: georgegevoian, dsagal
Reviewed By: georgegevoian
Subscribers: dsagal, paulfitz
Differential Revision: https://phab.getgrist.com/D4158
Summary:
Fixes a recent regression that would cause a record to be erroneously filtered
out whenever it was updated from a linked view.
Test Plan: Browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4163
Summary:
Adds initial implementation of form publishing, built upon WYSIWYS shares.
A simple UI for publishing and unpublishing forms is included.
Test Plan: Browser tests.
Reviewers: jarek
Reviewed By: jarek
Subscribers: paulfitz, jarek
Differential Revision: https://phab.getgrist.com/D4154
Summary:
- Move CellValue documentation to CellValue and add an example for each type.
- Link to CellValue from places that mention it.
- Update deprecated [[LINK]] syntax to a supported {@link} one, in a way that
does not change generated documentation.
- Also fix auto-pick-ports script used in tests (which had a bug causing occasional test failures), and add a test for it.
Test Plan: No code changes for documentation changes. New test for auto-pick-ports.
Reviewers: georgegevoian
Reviewed By: georgegevoian
Differential Revision: https://phab.getgrist.com/D4162
Summary:
adds a newly-added migration to a list in a test for explicit
rollback testing
Test Plan: updates a test
Reviewers: georgegevoian
Reviewed By: georgegevoian
Subscribers: georgegevoian
Differential Revision: https://phab.getgrist.com/D4161
Summary:
When linking using a Reference List column, there may be multiple source
records that show the same target record. With this change, we remember those
(rather than just pick one that shows the target record).
Test Plan: Added a browser test.
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4140
Summary: Adds `@functools.lru_cache` to `ts_to_dt` and `ts_to_date` which are used in `_make_rich_value` in Date/DateTime columns, i.e. every time such a column is accessed in a formula. I noticed that these operations are surprisingly expensive while working on https://phab.getgrist.com/D4075. This is just an easy way to potentially significantly speed up certain docs and formulas.
Test Plan:
Put this code in an engine test case:
```
def test_(self):
self.apply_user_action(["AddTable", "Table1", [
{"id": "A", "type": "DateTime:America/Chicago"},
]])
self.add_records("Table1", ["A"], [
[i] for i in range(1000)
])
formula = "for _ in range(1000): $A\nreturn 1"
import time
start = time.time()
self.add_column(
"Table1", formula, type="Any", isFormula=True, formula=formula
)
elapsed = time.time() - start
print(f"Took {elapsed:.2f} for formula {formula}")
```
The time goes from ~10s to ~1s. Similar tests showed no noticeable slowdown when caching had no effect.
Reviewers: dsagal
Reviewed By: dsagal
Differential Revision: https://phab.getgrist.com/D4099
Summary:
While adding `FetchSelectedOptions` in 4e67c679b2, I accidentally made `viewApi` become an undocumented variable, which is fixed here.
While I was at it, I also fixed other errors emitted by `./build-plugin-api.sh`.
Test Plan: Built changes in https://github.com/gristlabs/grist-help/pull/297
Reviewers: paulfitz
Reviewed By: paulfitz
Differential Revision: https://phab.getgrist.com/D4155
Summary:
This is fixing some fall-out from some environmental changes after
another batch of tests were moved to grist-core.
Test Plan: this is fixing a test, no app changes
Reviewers: jarek
Reviewed By: jarek
Differential Revision: https://phab.getgrist.com/D4156
Summary:
This gives a mechanism for controlling access control within a document that is distinct from (though implemented with the same machinery as) granular access rules.
It was hard to find a good way to insert this that didn't dissolve in a soup of complications, so here's what I went with:
* When reading rules, if there are shares, extra rules are added.
* If there are shares, all rules are made conditional on a "ShareRef" user property.
* "ShareRef" is null when a doc is accessed in normal way, and the row id of a share when accessed via a share.
There's no UI for controlling shares (George is working on it for forms), but you can do it by editing a `_grist_Shares` table in a document. Suppose you make a fresh document with a single page/table/widget, then to create an empty share you can do:
```
gristDocPageModel.gristDoc.get().docData.sendAction(['AddRecord', '_grist_Shares', null, {linkId: 'xyz', options: '{"publish": true}'}])
```
If you look at the home db now there should be something in the `shares` table:
```
$ sqlite3 -table landing.db "select * from shares"
+----+------------------------+------------------------+--------------+---------+
| id | key | doc_id | link_id | options |
+----+------------------------+------------------------+--------------+---------+
| 1 | gSL4g38PsyautLHnjmXh2K | 4qYuace1xP2CTcPunFdtan | xyz | ... |
+----+------------------------+------------------------+--------------+---------+
```
If you take the key from that (gSL4g38PsyautLHnjmXh2K in this case) and replace the document's urlId in its URL with `s.<key>` (in this case `s.gSL4g38PsyautLHnjmXh2K` then you can use the regular document landing page (it will be quite blank initially) or API endpoint via the share.
E.g. for me `http://localhost:8080/o/docs/s0gSL4g38PsyautLHnjmXh2K/share-inter-3` accesses the doc.
To actually share some material - useful commands:
```
gristDocPageModel.gristDoc.get().docData.getMetaTable('_grist_Views_section').getRecords()
gristDocPageModel.gristDoc.get().docData.sendAction(['UpdateRecord', '_grist_Views_section', 1, {shareOptions: '{"publish": true, "form": true}'}])
gristDocPageModel.gristDoc.get().docData.getMetaTable('_grist_Pages').getRecords()
gristDocPageModel.gristDoc.get().docData.sendAction(['UpdateRecord', '_grist_Pages', 1, {shareRef: 1}])
```
For a share to be effective, at least one page needs to have its shareRef set to the rowId of the share, and at least one widget on one of those pages needs to have its shareOptions set to {"publish": "true", "form": "true"} (meaning turn on sharing, and include form sharing), and the share itself needs {"publish": true} on its options.
I think special shares are kind of incompatible with public sharing, since by their nature (allowing access to all endpoints) they easily expose the docId, and changing that would be hard.
Test Plan: tests added
Reviewers: dsagal, georgegevoian
Reviewed By: dsagal, georgegevoian
Subscribers: jarek, dsagal
Differential Revision: https://phab.getgrist.com/D4144
The error can often be fixed by just reloading the document with no need
to worry the document owners
For example, when the error message is: "interrupted by reconnect"
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
