Commit Graph

410 Commits

Author SHA1 Message Date
Jarosław Sadziński
3c219e05f6 (core) Removing the new menu flag
Summary: Enabling the `GRIST_NEW_COLUMN_MENU` flag by default and removing it.

Test Plan: Existing

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4098
2023-11-08 07:37:54 +01:00
George Gevoian
95cbbb8910 (core) Improve dark mode in tutorials
Summary:
Headings 4, 5, and 6 are now properly visible in dark mode. Additionally,
pre-formatted text and code blocks have improved styling in dark mode.

Test Plan: Manual.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4108
2023-11-06 13:06:13 -05:00
Paul Fitzpatrick
8053c81d02 (core) updates from grist-core 2023-11-06 08:20:57 -05:00
Florent
10822d3b86
getHostType: consider APP_DOC_INTERNAL_URL as native (#715)
The getHostType() now returns "native" when the host corresponds to the value of APP_DOC_INTERNAL_URL. T
While trying to scale, with a different internal and public URL for doc workers, and having configured the org to be specified in the path (GRIST_ORG_IN_PATH=true), the APP_DOC_INTERNAL_URL parameter was not treated as internal which made the connection between home server and doc workers impossible.

---------
https://github.com/gristlabs/grist-core/pull/715
Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-11-06 09:24:59 +01:00
George Gevoian
7a85aaa7a1 (core) Add new telemetry events
Summary: Adds a handful of new telemetry events, and makes a few tweaks to allow for better organization of telemetry.

Test Plan: Manual.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D4100
2023-11-01 10:49:33 -04:00
Paul Fitzpatrick
07bb90b5a6
allow bundled widgets to be hidden from dropdown, and nested (#714)
This makes a few refinements to bundling widgets:

  * A widget with `published: false` is not shown in the
    custom widget dropdown in the UI. This is so widgets
    can be bundled with the app for "native" use (like the
    calendar widget) without immediately resulting in an
    extra listing in the UI. (There are improvements we'd
    like to make to the UI to better communicate widget
    provenance and quality eventually, which would be a
    helpful alternative to just a binary flag.)

  * A relative path to the custom widget manifest is
    respected. This will make the bundling process marginally
    neater.
2023-10-30 21:13:21 -04:00
Paul Fitzpatrick
cc9a9ae8c5 (core) support for bundling custom widgets with the Grist app
Summary:
This adds support for bundling custom widgets with the Grist app, as follows:

 * Adds a new `widgets` component to plugins mechanism.
 * When a set of widgets is provided in a plugin, the html/js/css assets for those widgets are served on the existing untrusted user content port.
 * Any bundled `grist-plugin-api.js` will be served with the Grist app's own version of that file. It is important that bundled widgets not refer to https://docs.getgrist.com for the plugin js, since they must be capable of working offline.
 * The logic for configuring that port is updated a bit.
 * I removed the CustomAttachedView class in favor of applying settings of bundled custom widgets more directly, without modification on view.

Any Grist installation via docker will need an extra step now, since there is an extra port that needs exposing for full functionality. I did add a `GRIST_TRUST_PLUGINS` option for anyone who really doesn't want to do this, and would prefer to trust the plugins and have them served on the same port.

Actually making use of bundling will be another step. It'll be important to mesh it with our SaaS's use of APP_STATIC_URL for serving most static assets.

Design sketch: https://grist.quip.com/bJlWACWzr2R9/Bundled-custom-widgets

Test Plan: added a test

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D4069
2023-10-27 17:00:10 -04:00
George Gevoian
de33c5a3c6 (core) Add tests and tooltips to new Add Column menu
Summary: Adds tooltips to the menu and tests for recently-added functionality.

Test Plan: Browser tests.

Reviewers: JakubSerafin

Reviewed By: JakubSerafin

Subscribers: JakubSerafin

Differential Revision: https://phab.getgrist.com/D4087
2023-10-24 11:53:26 -04:00
Jarosław Sadziński
69d5ee53a8 (core) Treating API urls as external in cells
Summary:
Links for the API endpoints in a cell didn't work as they were interpreted as
internal routes. Now they are properly detected as external.

Test Plan: Added new test

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D4078
2023-10-24 08:55:08 +02:00
Jakub Serafin
2521db4c55 (core) New Columns Menu
Summary:
A menu to be shown when new colum button is added. It's give access to various diffrent shortcuts, like adding new column, unhiding existing ones, fast adding lookup columns or trigger one (authoriship or timestamp). Design document can be found here: https://grist.quip.com/CTgxAQv9Ghjt/Add-Columns-more-easily
To turn on this menu flag GRIST_NEW_COLUMN_MENU to 1

Test Plan: UI tests suite under nbrowser/GridViewNewColumnMenu.ts

Reviewers: jarek, georgegevoian

Reviewed By: georgegevoian

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4074
2023-10-13 22:35:36 +02:00
George Gevoian
1be1e5f647 (core) Move theme from ConfigNotifier to ThemeNotifier
Summary:
This reverts the behavior of onOptions, which had unintentionally
changed recently and no longer matched the API documentation.

Test Plan: Existing tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4064
2023-10-10 21:45:10 -04:00
Jarosław Sadziński
ad299f338a (core) Converting big number (9 digits or more) to date directly
Summary:
Interpret huge numbers (>8 digits) as timestamps when converting numeric column to date.
Convert date/date time columns to timestamp when converted from numeric/int column.

Test Plan: Updated

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: dsagal, alexmojaki

Differential Revision: https://phab.getgrist.com/D4030
2023-10-05 16:58:25 +02:00
Jarosław Sadziński
cce185956c (core) Delete my account button
Summary:
Adding new "Delete my account" button to the profile page that allows users to remove completely
their accounts as long as they don't own any team site.

Test Plan: Added

Reviewers: georgegevoian, paulfitz

Reviewed By: georgegevoian, paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4037
2023-09-27 14:49:23 +02:00
Dmitry S
2705d41c34 (core) Add timeouts to prevent ActiveDoc bad state during shutdown.
Summary:
Add two shutdown-related timeouts.

1. One is to limit the duration of any work that happens once shutdown
   begins. In particular, waiting for an update to current time could block
   indefinitely if the data engine is unresponsive. Such awaits are now
   limited to 5 seconds.

2. The other is to allow documents to get shutdown for inactivity even when
   some work takes forever. Certain work (e.g. applying user actions)
   generally prevents a document from shutting down while it's pending. This
   prevention is now limited to 5 minutes.

   Shutting down a doc while something is pending may break some
   assumptions, and lead to errors. The timeout is long to let us assume
   that the work is stuck, and that errors are better than waiting forever.

Other changes:
- Periodic ActiveDoc work (intervals) is now started when a doc finishes
  loading rather than in the constructor. The difference only showed up in
  tests which makes the intervals much shorter.

- Move timeoutReached() utility function to gutil, and use it for
  isLongerThan(), since they are basically identical. Also makes sure that the
  timer in these is cleared in all cases.

- Remove duplicate waitForIt implementation (previously had a copy in both
  test/server and core/test/server).

- Change testUtil.captureLog to pass messages to its callback, to allow asserts
  on messages within the callback.

Test Plan:
Added new unittests for the new shutdowns, including a replication
of a bad state that was possible during shutdown.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D4040
2023-09-26 15:32:49 -04:00
George Gevoian
273b976cab (core) Polish dark mode and remove beta tag
Summary:
Polishes support for dark mode and enables syncing with the OS theme
by default.

Test Plan: Manual.

Reviewers: JakubSerafin

Reviewed By: JakubSerafin

Subscribers: JakubSerafin

Differential Revision: https://phab.getgrist.com/D4041
2023-09-21 13:14:48 -04:00
George Gevoian
927e92e3e8 (core) Add tip for calendar widget configuration
Summary:
The tip is shown in the creator panel, in the subtab that lists the column
mapping configuration for the calendar widget. The panel now  automatically
opens the first time a calendar widget is added to a page (via the Add New menu).

Test Plan: Manual.

Reviewers: JakubSerafin

Reviewed By: JakubSerafin

Subscribers: JakubSerafin, jarek

Differential Revision: https://phab.getgrist.com/D4047
2023-09-20 11:10:58 -04:00
George Gevoian
4c25aa7d3d (core) Support dark mode in custom widgets
Test Plan: Manual.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4036
2023-09-19 16:08:21 -04:00
Paul Fitzpatrick
ffbf93b85f (core) updates from grist-core 2023-09-18 15:04:08 -04:00
Paul Fitzpatrick
bc6f550471
normalize logging of dates in siteUsage across databases (#675)
Date handling for SQLite and PostgreSQL is inconsistent. This makes
sure that dates in `siteUsage` logs are in a consistent format that
includes time zone information.
2023-09-13 14:44:04 -04:00
George Gevoian
40c5f7b738 (core) Add documentCreated telemetry event
Summary:
The event is triggered whenever a document is created, imported, or
duplicated.

Test Plan: Tested manually.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4035
2023-09-13 01:13:00 -04:00
George Gevoian
76e822eb23 (core) Add welcomeQuestionsSubmitted telemetry event
Summary:
The new event captures responses to the welcome questionnaire.

Responses are also still sent to the special Grist document configured with
the DOC_ID_NEW_USER_INFO variable.

Test Plan: Tested manually.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4034
2023-09-13 00:31:04 -04:00
Paul Fitzpatrick
18f7e255df (core) updates from grist-core 2023-09-11 10:00:39 -04:00
Florent
5ff79703b4
Introduce GRIST_ANON_PLAYGROUND variable #642 (#651)
* `GRIST_ANON_PLAYGROUND`: When set to 'false' deny anonymous users access to the home page
 * `GRIST_FORCE_LOGIN`: Much like `GRIST_ANON_PLAYGROUND` but don't support anonymous access at all (features like sharing docs publicly requires authentication)

---------

Co-authored-by: Florent FAYOLLE <florent.fayolle@beta.gouv.fr>
2023-09-08 09:05:52 -04:00
George Gevoian
3dadf93c98 (core) Add support for auto-copying docs on signup
Summary:
The new "copyDoc" query parameter on the login page sets a short-lived cookie, which is
then read when welcoming a new user to copy that document to their Home workspace, and
redirect to it. Currently, only templates and bare forks set this parameter.

A new API endpoint for copying a document to a workspace was also added.

Test Plan: Browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3992
2023-09-06 15:12:08 -04:00
Paul Fitzpatrick
6dab12f301 (core) updates from grist-core 2023-09-05 11:12:15 -04:00
George Gevoian
70feb336d9 (core) Add AI assistant usage banners
Summary:
Banners are now shown when there are low or no AI assistant
credits remaining.

Test Plan: Browser tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4018
2023-08-30 16:00:04 -04:00
Jakub Serafin
1bc97773c8 (core) fix for master not building - in gristUrls.ts imports was duplicated
Test Plan: none

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4021
2023-08-30 11:06:31 +02:00
Jakub Serafin
942fc96225 (core) custom widget appear as build-in widget
Summary: Added boilerplate code needed to create new wigets in "Add new" menu, that are wrapped around existing custom widgets. More details can be found here: https://grist.quip.com/larhAGRKyl6Z/Custom-widgets-in-Add-Widget-menu

Test Plan: nbowser tests added to verify if item in menu exits, if widget is rendered, and right side menu has widget selection and read access selection hided.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3994
2023-08-30 09:44:25 +02:00
Paul Fitzpatrick
8d687a7651
remove a log message about fetching URLs (#643)
Every fetch made from the client is logged to the console.
But this isn't really necessary, and is particularly confusing
in grist-static, where those fetches are virtualized.

Tests in grist-saas may need adjusting to remove the logger.
2023-08-29 08:49:25 -04:00
Jarosław Sadziński
b6a431dd58 (core) Cursor in custom widgets
Summary:
Adding a new method `setCursorPos` in the widget API, and a new configuration option for the ready message `allowSelectBy` that exposes custom widgets in the `Select by` dropdown.
With this, a custom widget can control the position of the linked widgets and is able to change the column in the creator panel.

Test Plan: Added new test. Existing tests should pass.

Reviewers: JakubSerafin

Reviewed By: JakubSerafin

Subscribers: JakubSerafin

Differential Revision: https://phab.getgrist.com/D3993
2023-08-29 09:19:52 +02:00
Florent
9dfebefc9b
Dump the rule for ACL formula warnings (#639)
When an ACL formula fails to be run, a warning is printed. However, it is painful to know which formula is concerned by the warning.

ACL: if RHS is null, return false for "in" and "not in"
2023-08-23 09:23:29 -04:00
Paul Fitzpatrick
b3a71374d1 (core) updates from grist-core 2023-08-21 09:15:53 -04:00
Paul Fitzpatrick
0be858c19d
allow AI Assistance to run against any chat-completion-style endpoint (#630)
This adds an ASSISTANT_CHAT_COMPLETION_ENDPOINT which can be used
to enable AI Assistance instead of an OpenAI API key. The assistant
then works against compatible endpoints, in the mechanical sense.
Quality of course will depend on the model. I found some tweaks
to the prompt that work well both for Llama-2 and for OpenAI's models,
but I'm not including them here because they would conflict with some
prompt changes that are already in the works.

Co-authored-by: Alex Hall <alex.mojaki@gmail.com>
2023-08-18 16:14:42 -04:00
Alex Hall
166312be3a (core) Add dialog with options to allow downloading without history or data
Summary:
{F74398}

Refactored the 'radio checkboxes' in the modal for deleting a page and reused them here.

The option to download as a template already existed in the server code but wasn't being exercised by the frontend. Also added an option to remove just the history, which is the main motivation for this diff.

Test Plan: Expanded the existing nbrowser test.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3999
2023-08-18 15:38:24 +02:00
Jarosław Sadziński
fad421b7c0 (core) Removing temporary pro site
Summary: Creating a pro team site after Stripe checkout. Previously a stub site was always created and never removed, even if a user cancels the checkout process, which resulted in multiple 'ghost' sites that can't be removed.

Test Plan: Updated and added

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3985
2023-08-16 08:11:26 +02:00
Philip Standt
9df62e3d81
Add appearance parameter to override theme preferences (#620) 2023-08-15 10:29:29 -07:00
George Gevoian
24bbf375f9 (core) Add earliestDocCreatedAt to telemetry
Summary: This tracks the earliest document creation time, if any, for each site.

Test Plan: Server tests.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D3991
2023-08-08 00:48:18 -07:00
Dmitry S
efb5fc63d7 (core) Change the label/nickname of the paid plan to Pro
Summary:
This is to match the pricing page, and to reduce confusion. The same
change was made to the description of the corresponding "pricing" object
on Stripe. The Stripe value is what's used in practice; this value is
actually only used as a fallback.

Test Plan: The code change does not affect tests, but several tests fixed after grist-core merge.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3990
2023-08-07 23:45:18 -04:00
CamilleLegeron
02841bd15c
Header colored (#581) 2023-08-07 11:01:35 -07:00
Jarosław Sadziński
6416994c22 (core) Import redesign
Summary: New UI design for incremental imports.

Test Plan: Updated

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3945
2023-08-04 14:59:55 +02:00
George Gevoian
05c15e4ec3 (core) Show tweaked formula in AI responses
Summary:
The formula that's used when the Apply button is clicked, and the formula that's
shown in responses from the Formula Assistant should now be the same. Previously, they
would differ slightly.

Test Plan: Server tests.

Reviewers: alexmojaki

Reviewed By: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3977
2023-08-04 01:31:31 -07:00
John Cant
e1df6039c2
REQUEST now supports POST (#588)
* REQUEST now supports POST
* Add extra flag for enabling REQUEST, also update README and comments

Co-authored-by: John Cant <a.jonncant@gmail.com>
Co-authored-by: Alex Hall <alex.mojaki@gmail.com>
2023-07-30 15:13:43 -04:00
George Gevoian
a77170c4bd (core) Tweak navbar, breadcrumbs, and sign-in buttons
Summary:
The changes are intended to smooth over some sharp edges when a signed-out user
is using Grist (particularly while on the templates site).

Test Plan: Browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3957
2023-07-26 22:26:55 -07:00
Alex Hall
bc54a6646e (core) Filter rows based on linked widgets when exporting view
Summary:
Fixes a problem reported here: https://community.getgrist.com/t/exporting-the-records-in-a-linked-view/2556/4

The download CSV/Excel link now contains an additional `linkingFilter` URL parameter containing JSON-encoded `filters` and `operations`. This object is originally created in the frontend in `LinkingState`, and previously it was only used internally in the frontend. It would make its way via `QuerySetManager` to `QuerySet.getFilterFunc` where the actual filtering logic happened. Now most of that logic has been moved to a similar function in `common`. The new function works with a new interface `ColumnGettersByColId` which abstract over the different ways data is accessed in the client and server in this context. There's no significant new logic in the diff, just refactoring and wiring.

Test Plan: Expanded two `nbrowser/SelectBy*.ts` test suites to also check the contents of a downloaded CSV in different linking scenarios.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3961
2023-07-26 21:49:52 +02:00
Alex Hall
3f71c9c488 (core) Cleanup: Remove unused AssistanceRequest.regenerate
Summary: Finish what was started in https://phab.getgrist.com/D3970

Test Plan: existing tests

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3972
2023-07-26 15:51:46 +02:00
Jakub Serafin
f7fdfab6bf (core) GET endpoint for webhooks returns now data in format {webhooks:[...]}
Summary:
Rework of endpoint GET  for webhooks to make it coherent with other endpoints. Now data should be return in {webhooks:[{id:"...",fields:{"..."}]} format

```
{
    "webhooks": [
        {
            "id": ...
            "fields": {
                "url": ...
                "unsubscribeKey": ...
                "eventTypes": [
                    "add",
                    "update"
                ],
                "isReadyColumn": null,
                "tableId": "...",
                "enabled": false,
                "name": "...",
                "memo": "..."
            },
            "usage": {
                "status": "idle",
                "numWaiting": 0,
                "lastEventBatch": null
            }
        },
        {
            "id": "...",
            "fields": {
                "url": "...",
                "unsubscribeKey": "...",
                "eventTypes": [
                    "add",
                    "update"
                ],
                "isReadyColumn": null,
                "tableId": "...",
                "enabled": true,
                "name": "...",
                "memo": "..."
            },
            "usage": {
                "status": "error",
                "numWaiting": 0,
                "updatedTime": 1689076978098,
                "lastEventBatch": {
                    "status": "rejected",
                    "httpStatus": 404,
                    "errorMessage": "{\"success\":false,\"error\":{\"message\":\"Alias 5a9bf6a8-4865-403a-bec6-b4ko not found\",\"id\":null}}",
                    "size": 49,
                    "attempts": 5
                },
                "lastSuccessTime": null,
                "lastFailureTime": 1689076978097,
                "lastErrorMessage": "{\"success\":false,\"error\":{\"message\":\"Alias 5a9bf6a8-4865-403a-bec6-b4ko not found\",\"id\":null}}",
                "lastHttpStatus": 404
            }
        }
    ]
}
```

Test Plan: new test added to check if GET data fromat is correct. Other tests fixed to handle changed endpoint.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3966
2023-07-26 11:36:24 +02:00
Alex Hall
391c8ee087 (core) Allow assistant to evaluate current formula
Summary:
Replaces https://phab.getgrist.com/D3940, particularly to avoid doing potentially unwanted things automatically.

Adds optional fields `evaluateCurrentFormula?: boolean; rowId?: number` to `FormulaAssistanceContext` (part of `AssistanceRequest`). When `evaluateCurrentFormula` is `true`, calls a new function `evaluate_formula` in the sandbox which computes the existing formula in the column (regardless of anything the AI may have suggested) and uses that to generate an additional system message which is added before the user's message. In theory this could be used in an interface where users ask why a formula doesn't work, including possibly a formula suggested by the AI. For now, it's only used in `runCompletion_impl.ts` for experimenting.

Also cleaned up a bit, removing `_chatMode` which is always `true` now, and uses of `regenerate` which is always `false`.

Test Plan: Updated `runCompletion_impl` to optionally use the new feature, in which case it now scores 51/68 instead of 49/68.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3970
2023-07-24 21:59:00 +02:00
George Gevoian
90cf018133 (core) Update assistant link
Summary: Updates the "Learn more" link now that the page is published.

Test Plan: N/A

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3969
2023-07-21 13:43:31 -04:00
George Gevoian
beffd02c41 (core) Improve highlighting of previewed formula
Summary:
Also improves highlighting of columns when the "Click to insert"
tooltip is shown, and improves highlighting of transforming columns.

Test Plan: Manual.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3962
2023-07-20 14:34:22 -04:00
George Gevoian
0a34292536 (core) Add telemetry for AI Assistant
Summary: Also fixes a few bugs with some telemetry events not being recorded.

Test Plan: Manual.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3960
2023-07-20 12:50:26 -04:00