Dmitry S 9d4eeda480 (core) Python optimizations to speed up data engine ...

Summary:
- A bunch of optimizations guided by python profiling (esp. py-spy)
- Big one is optimizing Record/RecordSet attribute access
- Adds tracemalloc printout when running test_replay with PYTHONTRACEMALLOC=1 (on PY3)
  (but memory size is barely affected by these changes)

- Testing with RECORD_SANDBOX_BUFFERS_DIR, loading and calculating a particular
  very large doc (CRM), time taken improved from 73.9s to 54.8s (26% faster)

Test Plan: No behavior changes intended; relying on existing tests to verify that.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3781

2023-02-09 12:49:58 -05:00

..

functions

(core) Restoring separated transform columns

2023-01-03 17:54:16 +01:00

imports

Be more accepting with None value in headers candidate (#331)

2022-10-31 15:57:26 -04:00

acl_formula.py

(core) Support for $ syntax in ACL rules

2022-11-09 16:33:11 +01:00

acl.py

(core) Remaining Python 3 compatibility changes

2021-06-25 12:00:58 +02:00

action_obj.py

(core) External requests

2022-06-17 21:53:20 +02:00

action_summary.py

(core) Fix undo error for automatically removed rows, especially in summary tables

2022-09-09 22:15:45 +02:00

actions.py

(core) AddOrUpdateRecord user action

2022-02-03 16:22:51 -05:00

autocomplete_context.py

(core) Show example values in formula autocomplete

2022-09-28 19:42:36 +02:00

codebuilder.py

(core) Support for $ syntax in ACL rules

2022-11-09 16:33:11 +01:00

column.py

(core) Python optimizations to speed up data engine

2023-02-09 12:49:58 -05:00

csv_patch.py

(core) Simple Python 3 compatibility changes

2021-06-22 17:13:17 +02:00

depend.py

(core) Implement trigger formulas (generalizing default formulas)

2021-06-25 22:53:07 -04:00

docactions.py

(core) Undo bug - restoring dependencies for trigger formulas

2021-12-06 16:09:41 +01:00

docmodel.py

(core) Comments

2022-10-17 13:38:21 +02:00

engine.py

(core) Show example values in formula autocomplete

2022-09-28 19:42:36 +02:00

formula_prompt.py

(core) Porting back AI formula backend

2023-02-08 17:15:59 +01:00

friendly_errors.py

(core) Upgrade friendly-traceback and its dependencies

2022-10-28 14:44:14 +02:00

gencode.py

(core) Error explanations from friendly-traceback

2022-08-12 19:45:00 +02:00

grist.py

(core) Use MixedTypesKey for sort_by arg of lookupRecords to avoid errors in Python 3

2021-11-09 18:08:29 +02:00

identifiers.py

(core) Remove accents when picking identifiers

2021-08-24 14:47:50 +02:00

import_actions.py

(core) Prevent rebuilding usercode for every AddColumn when importing

2022-05-24 22:29:20 +02:00

logger.py

(core) move data engine code to core

2020-07-29 08:57:25 -04:00

lookup.py

(core) Python optimizations to speed up data engine

2023-02-09 12:49:58 -05:00

main.py

(core) Porting back AI formula backend

2023-02-08 17:15:59 +01:00

match_counter.py

(core) move data engine code to core

2020-07-29 08:57:25 -04:00

migrations.py

(core) Add new UI for writing memos

2022-12-12 17:52:01 -05:00

moment.py

(core) New type conversion in the backend

2022-02-04 20:28:13 +02:00

objtypes.py

(core) Python optimizations to speed up data engine

2023-02-09 12:49:58 -05:00

parse_data.py

Add BooleanConverter to map proper boolean cells to a Bool column

2022-09-10 07:07:45 +02:00

records.py

(core) Python optimizations to speed up data engine

2023-02-09 12:49:58 -05:00

relabeling.py

(core) Simple Python 3 compatibility changes

2021-06-22 17:13:17 +02:00

relation.py

(core) Polish and enable Reference List widget

2021-08-12 11:51:21 -07:00

runtests.py

python(tests): print logging when running tests in verbose mode

2022-09-20 17:22:28 +02:00

sandbox.py

(core) communicate with sandbox via standard pipes

2021-07-12 06:45:47 -04:00

schema.py

(core) Add new UI for writing memos

2022-12-12 17:52:01 -05:00

sort_specs.py

(core) Adding sort options for columns.

2021-11-03 15:31:39 +01:00

summary.py

(core) Allow duplicating tables from Raw Data page

2022-09-29 08:59:42 -07:00

table_data_set.py

(core) Simple Python 3 compatibility changes

2021-06-22 17:13:17 +02:00

table.py

(core) Python optimizations to speed up data engine

2023-02-09 12:49:58 -05:00

test_acl_formula.py

(core) Support for $ syntax in ACL rules

2022-11-09 16:33:11 +01:00

test_acl_renames.py

(core) Update ACL resources/rules when tables/columns get renamed

2021-01-04 22:03:01 -05:00

test_actions.py

(core) move data engine code to core

2020-07-29 08:57:25 -04:00

test_codebuilder.py

(core) Error explanations from friendly-traceback

2022-08-12 19:45:00 +02:00

test_column_actions.py

(core) Nice summary table IDs

2022-07-14 12:09:56 +02:00

test_completion.py

(core) Adding new column in users table "ref" with unique identifier.

2022-10-04 15:19:28 +02:00

test_default_formulas.py

(core) Fix lookups in default formulas

2020-09-10 18:06:06 -04:00

test_depend.py

(core) Fix recursion error in invalidate_deps

2021-06-18 15:42:20 +02:00

test_derived.py

(core) Fix undo error for automatically removed rows, especially in summary tables

2022-09-09 22:15:45 +02:00

test_display_cols.py

(core) Automatically remove empty summary table rows

2022-07-08 18:56:41 +02:00

test_docmodel.py

(core) Backend restrictions for raw data widgets

2022-02-15 22:04:32 +02:00

test_engine.py

(core) Python optimizations to speed up data engine

2023-02-09 12:49:58 -05:00

test_find_col.py

Correct spelling mistakes

2022-02-19 09:46:49 +00:00

test_formula_error.py

(core) More helpful messages when formula probably needs to use Table.all

2022-08-24 14:49:33 +02:00

test_formula_prompt.py

(core) Porting back AI formula backend

2023-02-08 17:15:59 +01:00

test_formula_undo.py

(core) Mark column changing actions as indirect when adding data to empty column

2022-05-25 16:21:04 +02:00

test_functions.py

(core) Don't swallow TypeErrors in functions like SUM

2022-10-25 12:15:13 +02:00

test_gencode.py

(core) Error explanations from friendly-traceback

2022-08-12 19:45:00 +02:00

test_import_actions.py

(core) Create an extra raw data widget when creating a table

2022-02-01 21:19:30 +02:00

test_logger.py

(core) move data engine code to core

2020-07-29 08:57:25 -04:00

test_lookups.py

(core) Respect sort_by in lookupOne, and allow reverse sorting

2022-10-24 20:10:08 +02:00

test_match_counter.py

(core) Import Hashable from six.moves.collections_abc

2022-02-22 00:27:51 +02:00

test_migrations.py

(core) Simple Python 3 compatibility changes

2021-06-22 17:13:17 +02:00

test_moment.py

(core) New type conversion in the backend

2022-02-04 20:28:13 +02:00

test_record_func.py

(core) Simple Python 3 compatibility changes

2021-06-22 17:13:17 +02:00

test_recordlist.py

(core) Return RecordSet instead of list from property access when possible, to allow further property access

2022-04-05 18:05:00 +02:00

test_reflist_rel.py

(core) New type conversion in the backend

2022-02-04 20:28:13 +02:00

test_relabeling.py

(core) Remaining Python 3 compatibility changes

2021-06-25 12:00:58 +02:00

test_renames2.py

Correct spelling mistakes

2022-02-19 09:46:49 +00:00

test_renames.py

(core) Adding new column in users table "ref" with unique identifier.

2022-10-04 15:19:28 +02:00

test_replay.py

(core) Python optimizations to speed up data engine

2023-02-09 12:49:58 -05:00

test_requests.py

(core) External requests

2022-06-17 21:53:20 +02:00

test_rules_grid.py

(core) Implementing row conditional formatting

2022-08-09 20:11:36 +02:00

test_rules.py

(core) Updating RawData views

2022-05-04 21:41:42 +02:00

test_side_effects.py

(core) Ensure formulas return something and don't assign to attributes of rec

2022-05-23 21:06:18 +02:00

test_sort_spec.py

(core) Adding sort options for columns.

2021-11-03 15:31:39 +01:00

test_summary2.py

(core) Test undo for all Python summary table tests

2022-09-07 18:02:02 +02:00

test_summary_choicelist.py

(core) Test undo for all Python summary table tests

2022-09-07 18:02:02 +02:00

test_summary_undo.py

(core) Nice summary table IDs

2022-07-14 12:09:56 +02:00

test_summary.py

(core) Fix undo error for automatically removed rows, especially in summary tables

2022-09-09 22:15:45 +02:00

test_table_actions.py

(core) Nice summary table IDs

2022-07-14 12:09:56 +02:00

test_table_data_set.py

(core) Simple Python 3 compatibility changes

2021-06-22 17:13:17 +02:00

test_temp_rowids.py

(core) Allow using negative rowIds to add records and refer to them in Reference values.

2020-12-15 16:39:56 -05:00

test_textbuilder.py

(core) Simple Python 3 compatibility changes

2021-06-22 17:13:17 +02:00

test_treeview.py

(core) move data engine code to core

2020-07-29 08:57:25 -04:00

test_trigger_formulas.py

(core) Restoring separated transform columns

2023-01-03 17:54:16 +01:00

test_twowaymap.py

(core) move data engine code to core

2020-07-29 08:57:25 -04:00

test_types.py

(core) Nicer conversion from numeric to text

2021-09-30 14:16:24 +02:00

test_undo.py

(core) Fix undo error for automatically removed rows, especially in summary tables

2022-09-09 22:15:45 +02:00

test_urllib_patch.py

(core) Remaining Python 3 compatibility changes

2021-06-25 12:00:58 +02:00

test_user.py

(core) Adding new column in users table "ref" with unique identifier.

2022-10-04 15:19:28 +02:00

test_useractions.py

(core) Update sort and filter UI

2022-11-17 15:33:45 -05:00

testsamples.py

(core) move data engine code to core

2020-07-29 08:57:25 -04:00

testscript.json

(core) Updating RawData views

2022-05-04 21:41:42 +02:00

testutil.py

(core) Implement trigger formulas (generalizing default formulas)

2021-06-25 22:53:07 -04:00

textbuilder.py

(core) Remaining Python 3 compatibility changes

2021-06-25 12:00:58 +02:00

treeview.py

(core) move data engine code to core

2020-07-29 08:57:25 -04:00

twowaymap.py

(core) Simple Python 3 compatibility changes

2021-06-22 17:13:17 +02:00

tzdata.data

(core) apply some dependabot suggestions

2022-09-07 14:15:34 -04:00

urllib_patch.py

(core) Remaining Python 3 compatibility changes

2021-06-25 12:00:58 +02:00

user.py

(core) Adding new column in users table "ref" with unique identifier.

2022-10-04 15:19:28 +02:00

useractions.py

(core) Restoring separated transform columns

2023-01-03 17:54:16 +01:00

usercode.py

(core) Raise syntax errors that Python can format nicely to show the location

2021-09-24 15:07:09 +02:00

usertypes.py

(core) Python optimizations to speed up data engine

2023-02-09 12:49:58 -05:00

xmlrunner.py

(core) Run python unit tests again in python 3

2021-06-30 21:46:18 +02:00