7 Commits (c2070877b66accb9bcdefcd269388d8197ed02f2)

Author	SHA1	Message	Date
George Gevoian	6abe7d5827	(core) Use original column headers during imports ... Summary: When possible, the original column headers from imported files will now be used as the labels for Grist columns. This includes values that were previously invalid Grist column identifiers, such as those containing Unicode. Test Plan: Updated server and browser tests. Reviewers: jarek Reviewed By: jarek Differential Revision: https://phab.getgrist.com/D3261	3 years ago
Alex Hall	fa9e6eee88	(core) Create an extra raw data widget when creating a table ... Summary: This is the first step towards raw data views, merely adding metadata without any UI. Every 'normal' table now has a widget referenced by `rawViewSectionRef`. It has no parent view/page and cannot actually be viewed for now. The widget is created during the AddTable user action, and the migration creates a widget for existing tables. Test Plan: Many tests had to be updated, especially tests that listed all view sections and/or fields. Reviewers: jarek, dsagal Reviewed By: jarek Differential Revision: https://phab.getgrist.com/D3232	3 years ago
George Gevoian	08b1286f4f	(core) Add column matching to Importer ... Summary: The Importer dialog is now maximized, showing additional column matching options and information on the left, with the preview table shown on the right. Columns can be mapped via a select menu listing all source columns, or by clicking a formula field next to the menu and directly editing the transform formula. Test Plan: Browser tests. Reviewers: jarek Reviewed By: jarek Differential Revision: https://phab.getgrist.com/D3096	3 years ago
George Gevoian	e1780e4f58	(core) Migrate import code from data engine to Node ... Summary: Finishing imports now occurs in Node instead of the data engine, which makes it possible to import into on-demand tables. Merging code was also refactored and now uses a SQL query to diff source and destination tables in order to determine what to update or add. Also fixes a bug where incremental imports involving Excel files with multiple sheets would fail due to the UI not serializing merge options correctly. Test Plan: Browser tests. Reviewers: jarek Reviewed By: jarek Differential Revision: https://phab.getgrist.com/D3046	3 years ago
George Gevoian	8a7edb6257	(core) Enable incremental imports ... Summary: The import dialog now has an option to 'Update existing records', which when checked will allow for selection of 1 or more fields to match source and destination tables on. If all fields match, then the matched record in the destination table will be merged with the incoming record from the source table. This means the incoming values will replace the destination table values, unless the incoming values are blank. Additional merge strategies are implemented in the data engine, but the import dialog only uses one of the strategies currently. The others can be exposed in the UI in the future, and tweak the behavior of how source and destination values should be merged in different contexts, such as when blank values exist. Test Plan: Python and browser tests. Reviewers: paulfitz Reviewed By: paulfitz Subscribers: alexmojaki Differential Revision: https://phab.getgrist.com/D3020	3 years ago
Alex Hall	16f297a250	(core) Simple Python 3 compatibility changes ... Summary: Changes that move towards python 3 compatibility that are easy to review without much thought Test Plan: The tests Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2873	3 years ago
Paul Fitzpatrick	b82eec714a	(core) move data engine code to core ... Summary: this moves sandbox/grist to core, and adds a requirements.txt file for reconstructing the content of sandbox/thirdparty. Test Plan: existing tests pass. Tested core functionality manually. Tested docker build manually. Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2563	4 years ago