gristlabs_grist-core/sandbox/grist/imports
Alex Hall 42afb17e36 (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling
Summary:
Python 2 only needs to be supported for the sake of old documents and formulas. This doesn't apply to the separate sandboxes that parse files for imports. Using Python 3 only allows using newer libraries and library versions. In particular, the latest version of openpyxl doesn't support Python 2. This will also make it easier to make other similar changes in the future, such as replacing messytables with a modern library. See https://grist.slack.com/archives/C0234CPPXPA/p1661261829343999?thread_ts=1661260442.837959&cid=C0234CPPXPA

The latest openpyxl is better at handling a particular edge case with broken dates in Excel, but still doesn't quite do what we want, so we monkeypatch it. Discussion: https://grist.slack.com/archives/C02EGJ1FUCV/p1661440851911869?thread_ts=1661154219.515549&cid=C02EGJ1FUCV

Setting `preferredPythonVersion` to '3' in SafePythonComponent ensures that JS always creates import sandboxes that use Python 3. Within Python, a module used by all imports will raise an error in Python 2. Python unit tests of imports are now only run in Python 3, using the `load_tests` protocol of `unittest`.

Test Plan: Mostly existing tests. Added another strange date to the Excel fixture.

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3606
2022-09-02 16:27:34 +02:00
..
fixtures (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling 2022-09-02 16:27:34 +02:00
__init__.py (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
import_csv_test.py (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling 2022-09-02 16:27:34 +02:00
import_csv.py (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
import_json_test.py (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling 2022-09-02 16:27:34 +02:00
import_json.py (core) When importing JSON, create columns of type Numeric rather than Int 2022-03-30 09:54:35 -04:00
import_utils.py (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling 2022-09-02 16:27:34 +02:00
import_xls_test.py (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling 2022-09-02 16:27:34 +02:00
import_xls.py (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling 2022-09-02 16:27:34 +02:00
messytables_test.py (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling 2022-09-02 16:27:34 +02:00
register.py (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
test_imports.py (core) Run and test imports only in Python 3, upgrade openpyxl, fix weird date handling 2022-09-02 16:27:34 +02:00