gristlabs_grist-core/sandbox
Alex Hall 321019217d (core) Lossless imports
Summary:
- Removed string parsing and some type guessing code from parse_data.py. That logic is now implicitly done by ValueGuesser by leaving the initial column type as Any. parse_data.py mostly comes into play when importing files (e.g. Excel) containing values that already have types, i.e. numbers and dates.
- 0s and 1s are treated as numbers instead of booleans to keep imports lossless.
- Removed dateguess.py and test_dateguess.py.
- Changed what `guessDateFormat` does when multiple date formats work equally well for the given data, in order to be consistent with the old dateguess.py.
- Columns containing numbers are now always imported as Numeric, never Int.
- Removed `NullIfEmptyParser` because it was interfering with the new system. Its purpose was to avoid pointlessly changing a column from Any to Text when no actual data was inserted. A different solution to that problem was already added to `_ensure_column_accepts_data` in the data engine in a recent related diff.

Test Plan:
- Added 2 `nbrowser/Importer2` tests.
- Updated various existing tests.
- Extended testing of `guessDateFormat`. Added `guessDateFormats` to show how ambiguous dates are handled internally.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3302
2022-03-08 12:14:39 +02:00
..
docker (core) support python3 in grist-core, and running engine via docker and/or gvisor 2021-07-28 09:02:32 -04:00
grist (core) Lossless imports 2022-03-08 12:14:39 +02:00
gen_js_schema.py (core) Use GristObjCode in CellValue 2021-10-11 14:11:32 +02:00
install_tz.js (core) move data engine code to core 2020-07-29 08:57:25 -04:00
requirements3.txt (core) Upgrade chardet 2021-10-21 18:29:17 +02:00
requirements.txt (core) Upgrade chardet 2021-10-21 18:29:17 +02:00