gristlabs_grist-core

Commit Graph

Author	SHA1	Message	Date
Dmitry S	534615dd50	(core) Update logging in sandbox code, and log tracebacks as single log messages. Summary: - Replace logger module by the standard module 'logging'. - When a log message from the sandbox includes newlines (e.g. for tracebacks), keep those lines together in the Node log message. Previously each line was a different message, making it difficult to view tracebacks, particularly in prod where each line becomes a separate message object. - Fix assorted lint errors. Test Plan: Added a test for the log-line splitting and escaping logic. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3956	1 year ago
Alex Hall	89259371a5	(core) Respect sort_by in lookupOne, and allow reverse sorting Summary: Ensure that `lookupOne` (via `RecordSet.get_one`) pays attention to the `sort_by` parameter by picking the first of its sorted list of row IDs. Allow specifying reverse sort order in `sort_by` by adding `"-"` before the column ID. Suggested in https://grist.slack.com/archives/C0234CPPXPA/p1665756041063079 Test Plan: Extended Python lookup test Reviewers: jarek Reviewed By: jarek Differential Revision: https://phab.getgrist.com/D3675	2 years ago
Alex Hall	437d30bd9f	(core) Log number of rows in user tables in data engine Summary: Adds a method Table._num_rows using an empty lookup map column. Adds a method Engine.count_rows which adds them all up. Returns the count after applying user actions to be logged by ActiveDoc. Test Plan: Added a unit test in Python. Tested log message manually. Reviewers: paulfitz Reviewed By: paulfitz Differential Revision: https://phab.getgrist.com/D3275	3 years ago
Alex Hall	0de0cb0f4a	(core) Add PUT /records DocApi endpoint to AddOrUpdate records Summary: As designed in https://grist.quip.com/fZSrAnJKgO5j/Add-or-Update-Records-API Current `POST /records` adds records, and `PATCH /records` updates them by row ID. This adds `PUT /records` to 'upsert' records, applying the AddOrUpdate user action. PUT was chosen because it's idempotent. Using a separate method (instead of inferring based on the request body) also cleanly separates validation, documentation, etc. The name `require` for the new property was suggested by Paul because `where` isn't very clear when adding records. Test Plan: New DocApi tests Reviewers: jarek Reviewed By: jarek Differential Revision: https://phab.getgrist.com/D3251	3 years ago
Dmitry S	f024aaaf5d	(core) Fix some bugs with ChoiceList in summary tables, and evaluation of lookups. Summary: Addresses several issues: - Error 'Cannot modify summary group-by column' when changing Text -> ChoiceList in the presence of summary tables. - Error 'ModifyColumn in unexpected position' when changing ChoiceList -> Text in the presence of summary tables. - Double-evaluation of trigger formulas in some cases. Fixes include: - Fixed verification that summary group-by columns match the underlying ones, and added comments to explain. - Avoid updating non-metadata lookups after each doc-action (early lookups generated extra actions to populate summary tables, causing the 'ModifyColumn in unexpected position' bug) - When updating formulas, do update lookups first. - Made a client-side tweak to avoid a JS error in case of some undos. Solution to reduce lookups is based on https://phab.getgrist.com/D3069?vs=on&id=12445, and tests for double-evaluation of trigger formulas are taken from there. Add a new test case to protect against bugs caused by incorrect order of evaluating #lookup columns. Enhanced ChoiceList browser test to check a conversion scenario in the presence of summary tables, previously triggering bugs. Test Plan: Various tests added or enhanced. Reviewers: alexmojaki Reviewed By: alexmojaki Subscribers: jarek Differential Revision: https://phab.getgrist.com/D3184	3 years ago
Alex Hall	267640c277	(core) Use MixedTypesKey for sort_by arg of lookupRecords to avoid errors in Python 3 Summary: title Test Plan: Added python unit test. Seems like the first test of sort_by in the whole codebase. Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D3124	3 years ago
Alex Hall	f7a9638992	(core) CONTAINS() and summarising by ChoiceList columns with flattening Summary: Added CONTAINS 'function' which can be used in lookups Changed LookupMapColumn._row_key_map to use right=set so one row can have many keys when CONTAINS is used. Use CONTAINS to implement group column in summary table, while helper column in source table can reference and create multiple rows in summary table, especially when summarising by ChoiceList columns. Use itertools.product to generate all combinations of lookup keys and groupby values. cleanup Test Plan: Added python unit tests. Reviewers: dsagal Reviewed By: dsagal Subscribers: paulfitz, dsagal Differential Revision: https://phab.getgrist.com/D2900	3 years ago
Alex Hall	a9d5b4d5af	(core) Refactor Table.Record[Set] classes Summary: Dealing with some things that bothered and sometimes confused me: Make Table.Record[Set] provide the table argument automatically Remove the classes from UserTable because they're not used anywhere and the Table/UserTable distinction is already confusing. They're not documented for users and they don't show up in autocomplete. Remove RecordSet.Record because it was confusing me where that attribute was being set, but also this means .Record will work properly for users with columns named 'Record'. Test Plan: existing tests Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2913	3 years ago
Dmitry S	e2226c3ab7	(core) Store formula values in DB, and include them into .stored/.undo fields of actions. Summary: - Introduce a new SQLiteDB migration, which adds DB columns for formula columns - Newly added columns have the special ['P'] (pending) value in them (in order to show the usual "Loading..." on the first load that triggers the migration) - Calculated values are added to .stored/.undo fields of user actions. - Various changes made in the sandbox to include .stored/.undo in the right order. - OnDemand tables ignore stored formula columns, replacing them with special SQL as before - In particular, converting to OnDemand table leaves stale values in those columns, we should maybe clean those out. Some tweaks on the side: - Allow overriding chai assertion truncateThreshold with CHAI_TRUNCATE_THRESHOLD - Rebuild python automatically in watch mode Test Plan: Fixed various tests, updated some fixtures. Many python tests that check actions needed adjustments because actions moved from .stored to .undo. Some checks added to catch situations previously only caught in browser tests. Reviewers: paulfitz Reviewed By: paulfitz Differential Revision: https://phab.getgrist.com/D2645	4 years ago
Paul Fitzpatrick	b82eec714a	(core) move data engine code to core Summary: this moves sandbox/grist to core, and adds a requirements.txt file for reconstructing the content of sandbox/thirdparty. Test Plan: existing tests pass. Tested core functionality manually. Tested docker build manually. Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2563	4 years ago

10 Commits (550c39156b944c705e423b6491ae0969620ce11b)