Commit Graph

947 Commits

Author SHA1 Message Date
Dmitry S
b722fb3aaa (core) If invalid tableIds are present in Access Rules, avoid browser errors to show the rules anyway
Summary:
- Previously a JS error was shown when an invalid table was present.
- Now invalid tables are shown as '#Invalid (tableId)'

Test Plan: Added a test case that Access Rules page can load after deletion.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3614
2022-08-31 11:02:56 -04:00
Dmitry S
5e0bf50574 (core) Update text in the delete-page dialog
Summary:
In the delete-page dialog, for the option to keep the data, use the somewhat
clearer text "Keep data and delete page".

Test Plan: No changes to testId, should not affect tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3612
2022-08-31 11:01:40 -04:00
Alex Hall
81810bd44b (core) Don't show doc/onboarding tour in embedded mode
Summary:
Tours don't really make sense when you can't see the full interface and document, so they're not shown when the interface style is 'light'.

Discussion: https://grist.slack.com/archives/CDHABLZJT/p1661459343666569?thread_ts=1661183887.200899&cid=CDHABLZJT

Test Plan: Extended nbrowser DocTour test

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3607
2022-08-30 16:46:56 +02:00
Alex Hall
1c43aed5dd (core) Upgrade sortedcontainers, python_dateutil, and html5lib for better Python 3.10 compatibility
Summary:
Based on https://github.com/gristlabs/grist-core/pull/251. It may not look like it, but there's very little going on in this diff:

- Tweak the DATEVALUE doctest for Python 2/3 compatibility.
- Mirrors the PR's changes to requirements3.txt in requirements.txt, i.e. make the same dependency upgrades in Python 2.
- Make the same upgrades in the thirdparty folder for the Python 2 nacl sandbox.

Test Plan: Updated one doctest for dateutil. Checked changelog of sortedcontainers. html5lib is only used by messytables and isn't actually relevant.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3609
2022-08-30 16:46:29 +02:00
Paul Fitzpatrick
63683f98cc (core) updates from grist-core 2022-08-26 17:29:25 -04:00
Alex Hall
ec1d0801fc
Merge pull request #251 from yohanboniface/py10
Update some python deps for python>=3.8 compat
2022-08-26 23:27:54 +02:00
Yohan Boniface
76fc6c87ae
Add docs/ and sandbox_venv*/ in .gitignore (#258)
docs/ is the default path where docs are storer when running app locally
sandbox_venv*/ are created when installing python
2022-08-26 16:20:01 -04:00
Yohan Boniface
ac9f2fbdd1 Update some python deps to prepare python>=3.10 compat
ABC classes have moved from collections to collections.abc since
python 3.3, and retro compat has ended with 3.8
2022-08-26 16:45:24 +02:00
Yohan Boniface
7ae425f869
add a description property to column mapping for custom widgets (#255) 2022-08-26 08:25:34 -04:00
Jarosław Sadziński
b6f5718ad0 (core) Fixing ACIndex highlightMatches functions
Summary:
Highlighting wasn't working correctly for the new
normalized search for autocomplate widgets.

Test Plan: Existing tests

Reviewers: alexmojaki

Reviewed By: alexmojaki

Differential Revision: https://phab.getgrist.com/D3602
2022-08-26 08:34:16 +02:00
George Gevoian
2cb783ea7b (core) Fix bugs with intervals
Summary:
Fixes some bugs involving intervals, and updates RandomizedTimer to support both fixed and
randomized delays, and to better handle async callbacks.

 * Fixed a bug where Throttle would queue up many pidusage calls due to the use of
    setInterval, and the async nature of the calls.

 * Fixed a but where RandomizedTimer (now just Interval) would not be disabled in
    ActiveDoc on doc shutdown if initialization had not yet settled.

Test Plan: Tested manually.

Reviewers: jarek, dsagal

Reviewed By: jarek, dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3604
2022-08-25 12:38:36 -07:00
Dmitry S
af77824618 (core) Add caching for measuring data size in DocStorage, when data isn't changing
Summary:
getDataSize() call can be expensive and involve lots of disk reading. We can
avoid doing it repeatedly when the document isn't actually changing.

Test Plan: Should have no change in behavior except for timings.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3605
2022-08-25 09:50:23 -04:00
Jarosław Sadziński
2997434815 (core) Showing a raw data section on a popup
Summary:
Show raw data will now open a popup with
raw section instead of redirecting to raw data page.

Adding new anchor link type "a2" that is able to open
any section in a popup on a current view.

Not related:
Fixing highlightMatches function, after merging core PR.

Test Plan: Updated tests

Reviewers: alexmojaki, georgegevoian

Reviewed By: alexmojaki, georgegevoian

Subscribers: georgegevoian, alexmojaki

Differential Revision: https://phab.getgrist.com/D3592
2022-08-25 09:30:07 +02:00
George Gevoian
1c722014fa
Merge pull request #254 from vviers/main
fix(ui): pointer cursor when creating a new widget
2022-08-24 14:40:48 -07:00
Yohan Boniface
50a57c673f
Add GRIST_DEFAULT_LOCALE env var (#257) 2022-08-24 15:24:50 -04:00
Paul Fitzpatrick
177b9d83d9 (core) add a log message on proxying failures
Summary:
When a home server fails to fetch from a doc worker, errors are
passed along to clients but we don't get to see them. This fixes
that omission.

Test Plan:
tested manually, by inserting some code to delay
serving particular test documents.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3601
2022-08-24 09:16:19 -04:00
Alex Hall
eac1f26f3e (core) More helpful messages when formula probably needs to use Table.all
Summary:
Raise an exception with a customised message for two cases when a user tries on operation directly on a table without `.all`:

1. For `Table.Col`, where `Col` is an existing column, suggest `Table.all.Col`. If `Col` doesn't exist as a column, fall back to the standard AttributeError.
2. When iterating directly over a table, e.g. `[r for r in Table]`, suggest looping over `Table.all` instead.

Test Plan: Added Python unit tests.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3593
2022-08-24 14:49:33 +02:00
Vincent Viers
2f48be26b1 fix(ui): add default cursor over disabled items 2022-08-24 09:47:47 +02:00
George Gevoian
56e8e1f4b3 (core) Randomize ActiveDoc interval delays
Summary:
When large numbers of documents were restarted simultaneously, they had
a tendency to schedule intervals to begin at roughly the same moment in
time, causing periodic spikes in load. This randomizes the delay of each
interval to help avoid such spikes.

Test Plan: Tested manually.

Reviewers: alexmojaki

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3600
2022-08-23 23:08:07 -07:00
George Gevoian
ed37401b2c (core) Add basic activation page to grist-ee
Summary:
Adds an activation page to grist-ee that currently shows activation status.

Follow-up diffs will introduce additional functionality, such as the ability to
enter activation keys directly from the activation page.

Test Plan: No grist-ee tests (yet).

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: dsagal, paulfitz

Differential Revision: https://phab.getgrist.com/D3582
2022-08-23 10:30:52 -07:00
Alex Hall
5f17dd0a06 (core) Convince grist-help that REQUEST is really unimplemented
Summary:
Change made to allow rebuilding functions in https://github.com/gristlabs/grist-help/pull/170

Deals the error described here: https://grist.slack.com/archives/C0234CPPXPA/p1660245534781629

Test Plan: Regenerating functions in grist-help works again.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3594
2022-08-23 19:25:27 +02:00
Jarosław Sadziński
a52d56f613 (core) Moving client and common tests to core
Summary:
- Moved /test/client and /test/common to core.
- Moved two files (CircularArray and RecentItems) from app/common to core/app/common.
- Moved resetOrg test to gen-server.
- `testrun.sh` is now invoking common and client test from core.
- Added missing packages to core's package.json (and revealed underscore as it is used in the main app).
- Removed Coord.js as it is not used anywhere.

Test Plan: Existing tests

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3590
2022-08-23 19:20:10 +02:00
Alex Hall
e06f0bc1d8 (core) Retry flaky daily API usage test
Summary: This particular test fails often enough to be annoying but not often enough to be worrying. It's not clear why, but it seems like a race condition involving redis. Fixing the test 'properly' seems hard and not worth the effort. Looking at the past 20 Jenkins builds, I found this test failing once. If we assume that the probability of failing is 1/20 (it's probably less since the test actually runs 3 times with different server configurations) then the probability of failing 3 times independently is (1/20)^3 = 1/8000, so `this.retry(3)` seems like a good enough solution. See also: https://grist.slack.com/archives/C0234CPPXPA/p1652909955773049

Test Plan: Manually made the test fail randomly 90% of the time, in which case `this.retries(3)` was not usually enough to prevent failures, but `this.retries(300)` was.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3595
2022-08-23 15:22:08 +02:00
Vincent Viers
5371ebcf46 fix(ui): pointer cursor when creating a new widget 2022-08-23 14:30:52 +02:00
Paul Fitzpatrick
028146f88a (core) updates from grist-core 2022-08-22 10:23:26 -04:00
Jarosław Sadziński
a43a5a25a6 (core) Removing old billing landing page.
Summary: Old landing page /docs/billing/signup is not used anymore.

Test Plan: Updated tests

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3585
2022-08-22 07:37:13 +02:00
Alex Hall
42060df29a (core) Formula autocomplete improvements for references and lookups
Summary:
Makes the following improvements to formula autocomplete:

- When a user types `$RefCol` (or part of it), also show `$RefCol.VisibleCol` (replace actual column names) in the autocomplete even before the `.` is typed, to help users understand the difference between a raw reference/record and its visible column.
- When a user types a table name, show `.lookupOne` and `.lookupRecords` in the autocomplete, again even before the `.` is typed.
- For `.lookupRecords(` and `.lookupOne(`, once the `(` is entered, suggest each column name as a keyword argument.
- Also suggest lookup arguments involving compatible reference columns, especially 'reverse reference' lookups like `refcol=$id` which are very common and difficult for users.
- To support these features, the Ace editor autocomplete needs some patching to fetch fresh autocomplete options after typing `.` or `(`. This also improves unrelated behaviour that wasn't great before when one column name is contained in another. See the first added browser test.

Discussions:

- https://grist.slack.com/archives/CDHABLZJT/p1659707068383179
- https://grist.quip.com/HoSmAlvFax0j#MbTADAH5kgG
- https://grist.quip.com/HoSmAlvFax0j/Formula-Improvements#temp:C:MbT3649fe964a184e8dada9bbebb

Test Plan: Added Python and nbrowser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3580
2022-08-20 19:11:41 +02:00
Jarosław Sadziński
44b4ec7edf (core) Removing duplicated tests
Summary: Removing duplicated tests that were moved to grist-core.

Test Plan: Existing tests

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D3588
2022-08-19 10:29:53 +02:00
jarek
4554de661e
Merge pull request #240 from ronnix/ignore-diacritics-in-autocomplete
Ignore diacritics in autocomplete
2022-08-18 10:49:16 +02:00
Paul Fitzpatrick
b286256f96
add link to new self-managed handbook (#246)
Adds a link to a new page for documenting configuration of self-managed Grist.
2022-08-17 19:07:48 -04:00
Jarosław Sadziński
758c87f002 (core) Adding hide for multiple columns
Summary: New column menu option to hide multiple columns.

Test Plan: new test

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3581
2022-08-17 22:31:16 +02:00
Ronan Amicel
cd16650e0e Rename text cleaning function to "normalizeText" 2022-08-17 19:11:52 +02:00
Ronan Amicel
4003d2d7c6 Use lodash to remove accents and ther diacritics
https://lodash.com/docs/4.17.15#deburr
2022-08-17 19:11:52 +02:00
Ronan Amicel
0249b3d31b Add tests for autocomplete without accents 2022-08-17 19:11:52 +02:00
George Gevoian
360d838578 (core) Add Open Graph image tags
Summary: Adds a few missing Open Graph tags to Grist pages.

Test Plan: Manual.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3586
2022-08-16 10:54:32 -07:00
Ronan Amicel
2a05d04e35 Ignore diacritics in autocomplete
Works for:
- Choice
- Choice List
- Reference
- Reference List

Co-Authored-By: Louis Delbosc <louis.delbosc.prestataire@anct.gouv.fr>
2022-08-16 09:20:08 +02:00
George Gevoian
0c5441b176 (core) Add unquarantine command to admin CLI
Summary:
Adds a CLI command to un-quarantine an active document. Also tweaks the
name of related environment variable to avoid a naming conflict.

Test Plan: Server test.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3583
2022-08-15 13:04:55 -07:00
Paul Fitzpatrick
103c795aa2 (core) make sure grist core yarn lockfile is tested
Summary:
This tweaks the grist core testing in our monorepo to match build
conditions for grist core docker images. It updates the grist core
yarn.lock which has become a little stale.

Test Plan: updated test

Reviewers: georgegevoian

Reviewed By: georgegevoian

Subscribers: georgegevoian

Differential Revision: https://phab.getgrist.com/D3584
2022-08-15 14:44:52 -04:00
Paul Fitzpatrick
e95b215405 (core) updates from grist-core 2022-08-15 10:00:08 -04:00
jarek
e90f48c551
Merge pull request #243 from gristlabs/skip-external-tests
Skipping tests that can't be run on external server.
2022-08-12 20:28:54 +02:00
Alex Hall
49cb51bac5 (core) Error explanations from friendly-traceback
Summary: Extend formula error messages with explanations from https://github.com/friendly-traceback/friendly-traceback. Only for Python 3.

Test Plan: Updated several Python tests. In general, these require separate branches for Python 2 and 3.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3542
2022-08-12 19:45:00 +02:00
Jarosław Sadziński
22ed7d3024 Skipping tests that can't be run on external server. 2022-08-12 17:23:22 +02:00
Alex Hall
31f54065f5 (core) Use standard library uuid.uuidv4 when possible for better randomness
Summary: Using the `random` module in the Grist `UUID()` function is not cryptographically secure, and is only necessary for the old pynbox (Python 2) sandbox which doesn't support `os.urandom`. This diff uses the `uuid.uuidv4()` function from the Python standard library when possible, which is more secure, only falling back to the old implementation when necessary.

Test Plan: Added Python unit tests to check both implementations.

Reviewers: dsagal

Subscribers: paulfitz, dsagal

Differential Revision: https://phab.getgrist.com/D3578
2022-08-12 16:45:11 +02:00
George Gevoian
09db2253e4
Merge pull request #242 from gristlabs/import-empty-columns-bug
Fix bug that skips empty columns during imports
2022-08-11 12:08:07 -07:00
George Gevoian
ebcfd2074f Fix bug that skips empty columns during imports
A faulty conditional in _makeDefaultTransformRule was the cause of the
bug. The conditional isn't necessary, as it's unreachable from the
import flows, so it was removed.
2022-08-11 11:05:30 -07:00
George Gevoian
3ad78590c2 (core) Polish display of table row counts
Summary:
Adds a new dots loader, which is used on the Raw Data page when
certain values are still being calculated (e.g. row counts). Now metrics
whose values aren't known yet will still appear under Usage, but with a
"Loading" message and the dots loader shown. For per-table row counts,
we only show the loader.

Test Plan: Existing tests.

Reviewers: jarek

Reviewed By: jarek

Subscribers: jarek

Differential Revision: https://phab.getgrist.com/D3566
2022-08-11 09:45:33 -07:00
Alex Hall
b416a5c4b1 (core) Fix error when updating summary table formulas after rename
Summary:
Formulas in summary tables were being associated with the source table for automatic updating. When a table/column was renamed such that the formula needed to update to match, it would look for a column with the same colId but in the source table. Such a column might not exist which would lead to an error, or if it existed then the update would be wrong.

This association was created while building formulas to display in the code view in a nested `_Summary` class, it didn't need to exist at all. So this diff simply prevents the association from being created.

User report and discussion: https://grist.slack.com/archives/C0234CPPXPA/p1659717322297019

Test Plan: Extended `TestSummary.test_table_rename` Python test.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3568
2022-08-11 12:51:11 +02:00
jarek
7f74652b26
Merge pull request #241 from gristlabs/moving-tests-to-grist-core
Moving more test to grist-core
2022-08-10 17:24:18 +02:00
Jarosław Sadziński
a44a59e591 Tweaking logging in github action and test_under_docker script 2022-08-10 14:16:27 +02:00
Jarosław Sadziński
c7a6784873 Removing grist logs from tests 2022-08-10 13:33:08 +02:00