gristlabs_grist-core/sandbox/grist
Alex Hall 9b36fb4dab (core) Fix error in sandbox when removing multiple summary source columns
Summary:
Fixes a very specific bug reported here: https://grist.slack.com/archives/C069RUP71/p1694630242765769

The error occurred when:

1. Removing multiple columns simultaneously
2. Those columns were sources of groupby columns for a summary table (so removing them meant recreating the summary table and thus deleting its columns)
3. There was a display column for one of the columns that got deleted (either directly or indirectly) which was set to be automatically removed since it was no longer needed, but this failed because the column was already deleted as part of earlier table removal.

I fixed this by making `apply_auto_removes` remove table records last, so removing the display column wouldn't be a problem.

That fixed the original error, but then I noticed that trying to undo the removal could lead to another error (under even more specific circumstances). It's hard to see exactly why, but I can see that just 3 `RemoveColumn` user actions generated over 100 doc actions and corresponding undo actions, hence the difficulty in narrowing the problem down. This is partly because removing a single column would recreate a summary table, only for that table to be immediately replaced again when another column was removed. Making the frontend send a single `[BulkRemoveRecord, _grist_Tables_column, ...] ` leads to a more efficient and sensible process with about half as many doc actions and no undo error. I think this alone would also solve the original error, but the data engine change seems more generally helpful and worth keeping.

Test Plan: Added a Python test and an nbrowser test

Reviewers: jarek

Reviewed By: jarek

Differential Revision: https://phab.getgrist.com/D4052
2023-09-27 16:23:49 +02:00
..
functions REQUEST with unknown and undetectable encoding (#600) 2023-08-02 16:49:37 -04:00
imports (core) Improve encoding detection for csv imports, and make encoding an editable option. 2023-08-24 09:50:52 -04:00
acl_formula.py (core) Support for $ syntax in ACL rules 2022-11-09 16:33:11 +01:00
acl.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
action_obj.py
action_summary.py (core) Fix undo error for automatically removed rows, especially in summary tables 2022-09-09 22:15:45 +02:00
actions.py
attribute_recorder.py (core) Allow assistant to evaluate current formula 2023-07-24 21:59:00 +02:00
autocomplete_context.py (core) Make Python tests pass in Python 3.11 2023-07-28 16:37:38 +02:00
codebuilder.py (core) Fix updating attributes inside f-strings when columns are renamed 2023-09-11 14:41:30 +02:00
column.py (core) Fix undo of Ref->Numeric conversion. 2023-09-27 08:53:53 -04:00
csv_patch.py
depend.py
docactions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
docmodel.py (core) Fix error in sandbox when removing multiple summary source columns 2023-09-27 16:23:49 +02:00
engine.py (core) Allow assistant to evaluate current formula 2023-07-24 21:59:00 +02:00
fake_std_streams.py (core) Use fake stdout/stderr when evaluating formulas 2023-06-14 18:24:27 -04:00
formula_prompt.py (core) Use new asttokens.ASTText to support dollar signs inside f-strings 2023-08-23 13:39:28 +02:00
friendly_errors.py (core) Upgrade friendly-traceback and its dependencies 2022-10-28 14:44:14 +02:00
gencode.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
grist.py
identifiers.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
import_actions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
lookup.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
main.py (core) Allow assistant to evaluate current formula 2023-07-24 21:59:00 +02:00
match_counter.py
migrations.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
moment.py
objtypes.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
parse_data.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
records.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
relabeling.py
relation.py
runtests.py python(tests): print logging when running tests in verbose mode 2022-09-20 17:22:28 +02:00
sandbox.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
schema.py (core) reconcile webhook and widget description migrations 2023-05-15 11:56:15 -04:00
sort_specs.py
summary.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
table_data_set.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
table.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_acl_formula.py (core) Support for $ syntax in ACL rules 2022-11-09 16:33:11 +01:00
test_acl_renames.py
test_actions.py
test_codebuilder.py Skip tests involving f-strings for Python 3.9 (#669) 2023-09-11 18:49:30 +02:00
test_column_actions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_completion.py (core) Make Python tests pass in Python 3.11 2023-07-28 16:37:38 +02:00
test_default_formulas.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_depend.py
test_derived.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_display_cols.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_docmodel.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_engine.py (core) Make Python tests pass in Python 3.11 2023-07-28 16:37:38 +02:00
test_find_col.py
test_formula_error.py (core) Ignore leading whitespace in formulas, and strip out leading '=' sign users might add 2023-04-25 15:28:40 -04:00
test_formula_prompt.py Skip tests involving f-strings for Python 3.9 (#669) 2023-09-11 18:49:30 +02:00
test_formula_undo.py
test_functions.py (core) Don't swallow TypeErrors in functions like SUM 2022-10-25 12:15:13 +02:00
test_gencode.py (core) Make Python tests pass in Python 3.11 2023-07-28 16:37:38 +02:00
test_import_actions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_lookups.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_match_counter.py
test_migrations.py
test_moment.py
test_record_func.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_recordlist.py
test_reflist_rel.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_relabeling.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_renames2.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_renames.py Skip tests involving f-strings for Python 3.9 (#669) 2023-09-11 18:49:30 +02:00
test_replay.py (core) Python optimizations to speed up data engine 2023-02-09 12:49:58 -05:00
test_requests.py REQUEST with unknown and undetectable encoding (#600) 2023-08-02 16:49:37 -04:00
test_rules_grid.py
test_rules.py
test_side_effects.py
test_sort_spec.py
test_summary2.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_summary_choicelist.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_summary_undo.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_summary.py (core) Fix error in sandbox when removing multiple summary source columns 2023-09-27 16:23:49 +02:00
test_table_actions.py (core) Keep referencing columns when their table is deleted, convert to appropriate type 2023-09-11 14:15:54 +02:00
test_table_data_set.py
test_temp_rowids.py
test_textbuilder.py
test_treeview.py
test_trigger_formulas.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_twowaymap.py
test_types.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
test_undo.py (core) Fix undo error for automatically removed rows, especially in summary tables 2022-09-09 22:15:45 +02:00
test_urllib_patch.py
test_user.py (core) Adding new column in users table "ref" with unique identifier. 2022-10-04 15:19:28 +02:00
test_useractions.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
testsamples.py
testscript.json (core) Keep referencing columns when their table is deleted, convert to appropriate type 2023-09-11 14:15:54 +02:00
testutil.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
textbuilder.py
treeview.py
twowaymap.py
tzdata.data (core) apply some dependabot suggestions 2022-09-07 14:15:34 -04:00
urllib_patch.py
user.py (core) Adding new column in users table "ref" with unique identifier. 2022-10-04 15:19:28 +02:00
useractions.py (core) Keep referencing columns when their table is deleted, convert to appropriate type 2023-09-11 14:15:54 +02:00
usercode.py (core) Make Python tests pass in Python 3.11 2023-07-28 16:37:38 +02:00
usertypes.py (core) Update logging in sandbox code, and log tracebacks as single log messages. 2023-07-18 11:21:25 -04:00
xmlrunner.py