gristlabs_grist-core/sandbox/grist/imports
Alex Hall 321019217d (core) Lossless imports
Summary:
- Removed string parsing and some type guessing code from parse_data.py. That logic is now implicitly done by ValueGuesser by leaving the initial column type as Any. parse_data.py mostly comes into play when importing files (e.g. Excel) containing values that already have types, i.e. numbers and dates.
- 0s and 1s are treated as numbers instead of booleans to keep imports lossless.
- Removed dateguess.py and test_dateguess.py.
- Changed what `guessDateFormat` does when multiple date formats work equally well for the given data, in order to be consistent with the old dateguess.py.
- Columns containing numbers are now always imported as Numeric, never Int.
- Removed `NullIfEmptyParser` because it was interfering with the new system. Its purpose was to avoid pointlessly changing a column from Any to Text when no actual data was inserted. A different solution to that problem was already added to `_ensure_column_accepts_data` in the data engine in a recent related diff.

Test Plan:
- Added 2 `nbrowser/Importer2` tests.
- Updated various existing tests.
- Extended testing of `guessDateFormat`. Added `guessDateFormats` to show how ambiguous dates are handled internally.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3302
2022-03-08 12:14:39 +02:00
..
fixtures (core) Fix import parsing from choking up on Python isdigit() surprises 2021-09-20 16:17:34 -04:00
__init__.py (core) move data engine code to core 2020-07-29 08:57:25 -04:00
import_csv.py (core) Move file import plugins into core/sandbox/grist 2021-08-09 18:37:14 +02:00
import_json.py Correct spelling mistakes 2022-02-19 09:46:49 +00:00
import_utils.py (core) Move file import plugins into core/sandbox/grist 2021-08-09 18:37:14 +02:00
import_xls.py (core) Move file import plugins into core/sandbox/grist 2021-08-09 18:37:14 +02:00
register.py (core) Move file import plugins into core/sandbox/grist 2021-08-09 18:37:14 +02:00
test_import_csv.py (core) Lossless imports 2022-03-08 12:14:39 +02:00
test_import_json.py (core) Move file import plugins into core/sandbox/grist 2021-08-09 18:37:14 +02:00
test_import_xls.py (core) Lossless imports 2022-03-08 12:14:39 +02:00
test_messytables.py (core) Simple Python 3 compatibility changes 2021-06-22 17:13:17 +02:00