Commit Graph

270 Commits

Author SHA1 Message Date
Paul Fitzpatrick
1af99e9567 (core) link AppSumo activations with stripe, and support upgrades/downgrades
Summary:
This links AppSumo sign-ups with Stripe subscriptions
and our billing pages. Different AppSumo tiers are supported by
different coupons on the standard plan. Configuration of this
is in stripe, and then cached in the database.

The front end is tweaked just enough to make completing a sign-up
possible. It is not yet friendly.

Not covered includes:
 * Streamlining landing page.
 * Making billing pages git clearer summaries of AppSumo states.
 * Making flow through Cognito as graceful as possible - default
   probably doesn't meet AppSumo requirements.
 * Disabling site on cancellation/refund.
 * Downgrades when more seats in use than lower tier allows.

Test Plan: api-level tests added. No front-end tests yet.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2878
2021-06-24 10:18:42 -04:00
Dmitry S
6240fd6982 (core) Fix reporting of errors when saving cells by clicking away, and deduplicate memos.
Test Plan: Enhanced the test case for memos to check these cases too (fails without this fix).

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2876
2021-06-23 08:50:42 -04:00
Cyprien P
7a0cd6c2b4 (core) Makes filter counts take other column filters into account
Summary:
Makes filter counts take other column filters into account.

 - Changes the summaries rows to reflect hidden rows:
    - hidden rows are added to the `Other Values` summary
    - show the unique number of other values as `Other Values (12)`
 - Also, adds a sort button to the column filter menu

Test Plan: Adds browser test.

Reviewers: paulfitz, jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D2861
2021-06-23 09:01:41 +02:00
Alex Hall
16f297a250 (core) Simple Python 3 compatibility changes
Summary: Changes that move towards python 3 compatibility that are easy to review without much thought

Test Plan: The tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2873
2021-06-22 17:13:17 +02:00
Paul Fitzpatrick
cc04c6481a (core) add appsumo endpoints with stub implementations
Summary:
This adds appsumo /token and /notification endpoints, with some
tests.  The stub implementation is sufficient for AppSumo
activation to succeed (when exposed via port forwarding for testing).
It needs fleshing out:

 * Implement upgrade/downgrade/refund and stripe subscription.
 * Implement custom landing page and flow.

Test Plan: added tests

Reviewers: dsagal, georgegevoian

Reviewed By: dsagal

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D2864
2021-06-21 16:04:33 -04:00
Alex Hall
ca57b3c099 (core) Updating dependencies only in preparation for Python 3 upgrade
Summary: Makes https://phab.getgrist.com/D2849 more manageable

Test Plan: None, could maybe run docs comparison

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2869
2021-06-18 15:43:02 +02:00
Alex Hall
746b26be19 (core) Fix recursion error in invalidate_deps
Summary: Convert recursion to while loop with stack of things to invalidate.

Test Plan: Loan tracker example works now

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2867
2021-06-18 15:42:20 +02:00
Jarosław Sadziński
bdd4d3c46e (core) Freezing columns on a GridView
Summary:
User can freeze any number of columns, which will not move when a user scrolls grid horizontally.
Main use cases:
- Frozen columns don't move when a user scrolls horizontally
- The number of frozen columns is automatically persisted
- Readonly viewers see frozen columns and can modify them - but the change is not persisted
- On a small screen - frozen columns still moves to the left when scrolled, to reveal at least one column
- There is a single menu option - Toggle freeze - which offers the best action considering selected columns
- When a user clicks a single column - action to freeze/unfreeze is always there
- When a user clicks multiple columns - action is offered only where it makes sens (columns are near the frozen border)

Test Plan: Browser tests

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2852
2021-06-18 12:22:13 +02:00
Jarosław Sadziński
698c9d4e40 (core) Readonly editors
Summary:
Grist should not prevent read-only viewers from opening cell editors since they usually provide much more information than is visible in a cell.

Every editor was enhanced with a read-only mode that provides the same information available for an editor but doesn't allow to change the underlying data.

Test Plan: Browser tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2842
2021-06-17 19:12:16 +02:00
Alex Hall
d3dc910784 (core) sys.setdefaultencoding('utf8')
Summary: sys.setdefaultencoding('utf8')

Test Plan: Will test against user documents to check for changes

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2858
2021-06-16 20:44:25 +02:00
Alex Hall
8a940676e9 (core) Generic tools for recording pycalls, deterministic mode.
Summary:
Replaces https://phab.getgrist.com/D2854

Refactoring of NSandbox:
- Simplify arguments to NSandbox.spawn. Only half the arguments were used depending on the flavour, adding a layer of confusion.
- Ensure the same environment variables are passed to both flavours of sandbox
- Simplify passing down environment variables.

Implement deterministic mode with libfaketime and a seeded random instance.
- Include static prebuilt libfaketime.so.1, may need another solution in future for other platforms.

Recording pycalls:
- Add script recordDocumentPyCalls.js to open a single document outside of tests.
- Refactor out recordPyCalls.ts to support various uses.
- Add afterEach hook to save all pycalls from server tests under $PYCALLS_DIR
- Make docTools usable without mocha.
- Add useLocalDoc and loadLocalDoc for loading non-fixture documents

Test Plan:
Made a document with formulas NOW() and UUID()
Compare two document openings in normal mode:

    diff <(test/recordDocumentPyCalls.js samples/d4W6NrzCMNVSVD6nWgNrGC.grist /dev/stdout) \
         <(test/recordDocumentPyCalls.js samples/d4W6NrzCMNVSVD6nWgNrGC.grist /dev/stdout)

Output:

    <                 1623407499.58132,
    ---
    >                 1623407499.60376,
    1195c1195
    <               "B": "bd2487f6-63c9-4f02-bbbc-5c0d674a2dc6"
    ---
    >               "B": "22e1a4fd-297f-4b86-91a2-bc42cc6da4b2"

`export DETERMINISTIC_MODE=1` and repeat. diff is empty!

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2857
2021-06-15 20:58:05 +02:00
George Gevoian
f613b68a9e (core) Fix widget loading spinner title
Summary:
The loading spinner would always display
'Building Table widget' when creating or changing
a widget. This fixes the title to reflect the selected
widget type.

Test Plan:
Updated existing browser tests to verify the loading spinner
title includes the correct widget type.

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2863
2021-06-15 08:56:24 -07:00
George Gevoian
3af6dd46ea (core) Allow docs to be permanently deleted in icon view
Summary:
Previously, soft-deleted docs in icon view were still accessible from
the Trash and couldn't be permanently deleted.

Test Plan:
Improved the nbrowser test for deleting docs to verify that it can
be done in both view modes.

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2862
2021-06-15 08:33:47 -07:00
Jarosław Sadziński
30bbb6fa94 (core) Summaries test doesn't work in a browser mode
Summary: Summaries test works only in a headless mode. In browser mode it failes as section is to narrow.

Test Plan: n/a

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2859
2021-06-14 16:10:00 +02:00
Cyprien P
b740028e8f (core) Increases readability of column filter menu code
Summary: The `select-all` originally designed the `All` button but it unfortunately evolves into designating both `All` and `None`. Which is confusing. Replaces with the more general `bulk-action`.

Test Plan: Should not break anything.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2860
2021-06-14 09:10:55 +02:00
George Gevoian
b94eb107d4 (core) Use individual choices for filtering choice lists
Test Plan: Wrote unit and browser tests that verify new behavior.

Reviewers: paulfitz, dsagal

Reviewed By: dsagal

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D2855
2021-06-11 09:34:06 -07:00
Cyprien P
5d3a4b5b5b (core) Introduce tri-state checkboxes
Summary:
 - Takes advantage of native indeterminate state of html checkboxes
 - When an indeterminate checkbox is clicked it turns it into being not checked.

Test Plan: - Added test to projects/UI2018

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2846
2021-06-11 14:53:42 +02:00
George Gevoian
982711dbba (core) Update active view id when default view id changes
Summary:
This fixes a bug where deleting a page with the page id
missing from the URL would cause JS errors to be thrown.

Test Plan:
Verified manually in dev environment. Browser test added
that should hopefully replicate the repro steps and catch
any regressions.

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2856
2021-06-10 17:49:39 -07:00
Paul Fitzpatrick
6f02987d10 (core) allow undos to be partial, if access control prohibits some part of them
Summary:
This is a somewhat experimental change, that will implement permitted parts of an undo if not all parts are permitted.  This is in preparation for trigger columns, where it may become common for a change in a record resulting in a change to an automatic change to another that the user cannot edit directly.  How to undo such an action is somewhat unclear.  One option is to undo the permitted parts, and then the triggers can rerun.

The general case is a bit of a can of worms, and feels adjacent to merging/rebasing etc.

Oh: it would probably be important in general to communicate to the user that an undo was partial, but this diff doesn't do that.  It would need some new plumbing.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2839
2021-06-10 18:26:11 -04:00
Jarosław Sadziński
af76c11be6 (core) Cursor position observable on a GristDoc wasn't triggered when a view was changed.
Summary: Cursor position observable was created using GrainJS, but the fields it was using were created using knockout observables. In a result the cursor position wasn't recomputed when a view was changed or an active section was deleted.

Test Plan: Browser tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2850
2021-06-10 14:38:56 +02:00
Alex Hall
8138cc3123 (core) Make regression tests by recording pycalls when opening fixture docs
Summary: Records all sandbox pycall arguments and results, saves them to JSON when the environment variable UPDATE_REGRESSION_DATA is set, otherwise checks that they match the saved JSON.

Test Plan: This is tests

Reviewers: dsagal, paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2851
2021-06-09 22:08:29 +02:00
George Gevoian
8bf26a4433 (core) Add cell formatting options to reference columns
Test Plan:
Tested manually by enabling/disabling wrapping and changing alignment on reference columns. Existing test updated to check that
reference columns have cell formatting options available.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2853
2021-06-08 14:59:03 -07:00
Jarosław Sadziński
24dca831c0 (core) Renaming column by clicking away resulted in renaming different column.
Summary:
Bug summary: if in right bar user starts changing name of column, but then clicks on a different column name in table, THAT column will have its name changed.

This bug occurs because the save method is invoked by a blur event on a input field, which is triggered after all computed observables are calculated. Save method gets an observable to update, which by the time a blur event triggers, is changed to a new column.

The solution was to forcefully trigger the blur event as soon as possible - here by subscribing to the cursor position observable.

Test Plan: Browser tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2845
2021-06-08 20:27:06 +02:00
Alex Hall
2f3a0e0c7f (core) Showing censored values as a grey cell
Test Plan: Block read access to column A based on the condition rec.B == 1. Then setting B = 1 in a row makes the cell under A grey.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: paulfitz, dsagal

Differential Revision: https://phab.getgrist.com/D2828
2021-06-07 13:11:41 +02:00
Paul Fitzpatrick
2feef7f780 (core) avoid typeorm's .save() method for relation with multi-column primary key
Summary:
A recently added stress test ("deletes documents reasonably quickly" in removedAt.ts) is sporadically failing under postgres.  It looks like typeorm's .save() method is in some way unreliable when setting a table with multi-column primary keys, via a ManyToMany relation. This diff replaces the .save() with explicit inserts/deletes.

I modified _repairWorkspaceGuests recently, so thought that change might have been the problem. However under the stress test, failures occur as often in _repairOrgGuests (not changed recently) as in _repairWorkspaceGuests (changed recently).

For reference, see schema diagram at https://grist.quip.com/wWpRAMe058Nl/Home-DB (the table being updated is `group_users`).

Possibly related issue: https://github.com/typeorm/typeorm/issues/4122

Test Plan:
After this change, stress test runs well on postgres locally (no failure 70 iterations); before it would fail on postgres within 3 iterations typically.

Separately: I gave a test that failed a little more time to return, and confirmed it was no slower on average, so I think it was unrelated.

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D2848
2021-06-04 13:58:22 -04:00
Paul Fitzpatrick
29c2b35dcc (core) speed up a step in document deletion
Summary:
The `_repairWorkspaceGuests` method is slow for workspaces with large numbers of documents.  It makes a query that produces a lot of rows.  The query itself is tolerable, but TypeORM processing uses enough CPU to be a likely culprit in some production instability.  This diff splits the query into two pieces that are logically independent, but which when combined were resulting in the number of rows being the product of the two pieces.  Once split, there is also a where clause that can be applied to one of the pieces.

The purpose of the method is to add every user that a document within a workspace is shared with to a "guest" group of the workspace itself.  The design of "guest" groups is not ideal, but this diff leaves the design unchanged and is intended only to speed up operation.

Made some small tweaks to the timing of a flakey test, and temporarily recreated the `samples` directory removed in a previous diff (this is currently breaking tests badly on a fresh worker without a `samples` directory lying around)

Test Plan: added test; existing tests pass

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D2844
2021-06-02 16:06:26 -04:00
Paul Fitzpatrick
69ac8fb4b3 (core) updates from grist-core 2021-06-01 13:19:22 -04:00
Paul Fitzpatrick
ed2b18c88a
Merge pull request #24 from georgegevoian/improved-doc-renaming-icon-view
Improve renaming docs in icon view
2021-06-01 13:18:27 -04:00
Dmitry S
11a5e0be34 (core) Add Lock icon
Test Plan: Nothing to test yet.

Reviewers: alexmojaki, paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz, jarek

Differential Revision: https://phab.getgrist.com/D2833
2021-06-01 13:10:57 -04:00
George Gevoian
68c1a8ba96 Improve renaming docs in icon view 2021-05-31 19:48:32 -07:00
Cyprien P
3a139e77c8 (core) Prevents running simultaneous search in parallel inadvertendly
Summary:
- this is a core search code refactoring
- this diff should fix the js error that was happening when searching across pages.

Test Plan: Tested manually on dev's environment. Tests shows no regression and successfully fixes the js error.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2837
2021-05-28 15:48:21 +02:00
Paul Fitzpatrick
c6265335af (core) make SELF_HYPERLINK urls cleaner
Summary:
This cleans up a few things about SELF_HYPERLINK urls:

 * Use `urlId` rather than `docId`.
 * Correctly merge personal org subdomain.
 * In dev environment, use clearer port number.

Test Plan: updated test

Reviewers: alexmojaki, dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2841
2021-05-28 08:54:37 -04:00
Paul Fitzpatrick
37698f9cb5 (core) apply access control to code view
Summary:
Names of private tables and columns were leaking via Code View.
This plugs that leak.

Test Plan: adds test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2840
2021-05-27 17:52:23 -04:00
Jarosław Sadziński
96fee73b70 (core) Download as CSV button on sections
Summary: Adding "Download as CSV" button that exports filtred section data to csv

Test Plan: Browser tests

Reviewers: paulfitz, dsagal

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2830
2021-05-27 15:48:12 +02:00
Jarosław Sadziński
5c0494fe29 (core) Draft cells
Summary: Cells will remember their previous state when user pressed the escape key. Grist will offer a way to continue with the draft, by showing notification and a tooltip above the editor.

Test Plan: Browser tests were created

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2822
2021-05-25 21:14:49 +02:00
Cyprien P
8c6148dd9f (core) fix closing of the search bar
Summary:
   - Sending `ESCAPE` while hovering the options panel was not working
   - This is because of the keepExpanded flag which was set to true
   - Solution is to set the keepExpanded flag to false, prior to toggling the Menu

Test Plan:  - Tested manually

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2836
2021-05-25 16:46:30 +02:00
Alex Hall
7758cb830d (core) Prevent error when there are no visible columns
Test Plan: Set all columns to hidden and reloaded document.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2832
2021-05-25 14:53:33 +02:00
Dmitry S
d1c1416d78 (core) Add rules to eslint to better match our coding conventions.
Summary:
We used tslint earlier, and on switching to eslint, some rules were not
transfered. This moves more rules over, for consistent conventions or helpful
warnings.

- Name private members with a leading underscore.
- Prefer interface over a type alias.
- Use consistent spacing around ':' in type annotations.
- Use consistent spacing around braces of code blocks.
- Use semicolons consistently at the ends of statements.
- Use braces around even one-liner blocks, like conditionals and loops.
- Warn about shadowed variables.

Test Plan: Fixed all new warnings. Should be no behavior changes in code.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2831
2021-05-24 12:56:18 -04:00
Alex Hall
0890749d15 (core) Change user attribute from select to formula, especially to allow link keys
Test Plan: Created a user attribute under access rules, set the attribute to look up to user.LinkKey.e, confirmed that setting e_ in the URL modified access.

Reviewers: dsagal, paulfitz

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2824
2021-05-21 12:22:55 +02:00
Alex Hall
3a586a5f6c (core) Navigate hyperlinks in the same document without a page reload
Summary: Extract out function _onClickHyperlink

Test Plan: Made a table, formatted column as hyperlink, added values including a link to another page in the document, another document, and an external website, clicked on all the links and only the first one didn't open a new tab.

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2826
2021-05-21 12:21:27 +02:00
Cyprien P
db91d31416 (core) Search improvemement
Summary:
This diff implements the search improvement that are mentioned here https://grist.quip.com/j1biAmfGkbzV/Search-Improvements.

CAVEATS:

I've noticed a bit of a usability caveats: the tooltips overlap with the new `search all pages` checkbox, which requires user to move cursor away for a bit in order to be able to click the button.

{F28224}

I've experimented locally with tooltips showing on both sides of the arrows, but it overlaps with the cross icon so could also be an issue. I couldn't think of any clear simple alternative, probably not too big of an issue anyway.

Test Plan: Added new test.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2818
2021-05-20 09:07:41 +02:00
Jarosław Sadziński
79497a7e79 (core) Editor position wasn't restored on a long list
Summary: Editor position wasn't restored on a long list, where the rows haven't been shown yet (the scroll haven't happend yet).

Test Plan: Browser tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2823
2021-05-19 19:14:44 +02:00
Cyprien P
e3a957a715 (core) Fixes persistence of the aclAsUser_ parameters across navigation
Summary:
  - The parameter was inadvertently removed by userOverrideParams().
  - when passed a function to `urlState().setHref(...)` or `urlState().makeUrl(...)` it is important that the function does not mutate the state that it receives as argument.

Link to the related task: https://gristlabs.getgrist.com/doc/check-ins/p/5#a1.s9.r791.c19

Test Plan: Adds test of the persistence by slightly modifying existing nbrowser/AccessRules2 tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D2820
2021-05-19 09:51:38 +02:00
Dmitry S
8c2f0307e5 (core) Preserve configured choices when converting between Choice and ChoiceList types.
Summary:
For conversions between Choice and ChoiceList, it makes more sense to preserve
the list of choices than to re-parse it from data.

Reported by Anais. Creating Choices from parsing ChoiceList cell values was
particularly poor, resulting in choices like "L,Foo,Bar".

Test Plan: Added a test case

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D2819
2021-05-18 09:59:20 -04:00
Jarosław Sadziński
5f182841b9 (core) Document keeps track of latest cursor position and latest editor value and is able to restore them when it is reloaded.
Summary: Grist document, when reloaded, is able to restore the latest cursor position and the editor state.

Test Plan: Browser test were created.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: paulfitz

Differential Revision: https://phab.getgrist.com/D2808
2021-05-18 10:36:59 +02:00
Paul Fitzpatrick
f5e3a0a94d (core) updates from grist-core 2021-05-15 16:44:56 -04:00
Paul Fitzpatrick
8c733dd5f1
Merge pull request #22 from combinatorist/patch-1
typo in intro
2021-05-15 16:44:08 -04:00
Timothy John Perisho Eccleston
af2d8c56b0
typo in intro 2021-05-15 11:12:36 -05:00
Jarosław Sadziński
2c93eafa44 (core) Null or undefined value on the Integer / Numeric cell
Summary: Text editor for Integer and Numeric column was showing null or undefined when the underlying value was null.

Test Plan: Browser test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2817
2021-05-14 21:41:37 +02:00
Paul Fitzpatrick
15723d1300 (core) check for +S bit early for changes that include formulas
Summary:
Currently, to compute intermediate steps in a bundle, the bundle
is sent to the data engine to process.  Then, if the intermediate
steps break a rule, it is reverted.  One problem introduced by
checking permissions this late is that the data engine can be
exposed for formulas with python code by users who don't have the
right to change formulas.  This diff pre-checks cases that change
formulas.

Test Plan: added a test

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2816
2021-05-14 09:18:05 -04:00