gristlabs_grist-core/sandbox/grist/imports/register.py

def register_import_parsers(sandbox):
  def parse_csv(file_source, options):
    from imports.import_csv import parse_file_source
    return parse_file_source(file_source, options)

  sandbox.register("csv_parser.parseFile", parse_csv)

  def parse_excel(file_source, parse_options):
    # pylint: disable=unused-argument
    from imports.import_xls import import_file
    return import_file(file_source)

  sandbox.register("xls_parser.parseFile", parse_excel)

  def parse_json(file_source, parse_options):
    from imports.import_json import parse_file
    return parse_file(file_source, parse_options)

  sandbox.register("json_parser.parseFile", parse_json)
(core) Move file import plugins into core/sandbox/grist Summary: Move all the plugins python code into the main folder with the core code. Register file importing functions in the same main.py entrypoint as the data engine. Remove options relating to different entrypoints and code directories. The only remaining plugin-specific option in NSandbox is the import directory/mount, i.e. where files to be parsed are placed. Test Plan: this Reviewers: paulfitz Reviewed By: paulfitz Subscribers: dsagal Differential Revision: https://phab.getgrist.com/D2965 2021-08-09 14:51:43 +00:00			`def register_import_parsers(sandbox):`
			`def parse_csv(file_source, options):`
			`from imports.import_csv import parse_file_source`
			`return parse_file_source(file_source, options)`

			`sandbox.register("csv_parser.parseFile", parse_csv)`

			`def parse_excel(file_source, parse_options):`
(core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows Summary: Use openpyxl instead of messytables (which used xlrd internally) in import_xls.py. Skip empty rows since excel files can easily contain huge numbers of them. Drop support for xls files (which openpyxl doesn't support) in favour of the newer xlsx format. Fix some details relating to python virtualenvs and dependencies, as Jenkins was failing to find new Python dependencies. Test Plan: Mostly relying on existing tests. Updated various tests which referred to xls files instead of xlsx. Added a Python test for skipping empty rows. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3406 2022-05-06 15:26:45 +00:00			`# pylint: disable=unused-argument`
(core) Move file import plugins into core/sandbox/grist Summary: Move all the plugins python code into the main folder with the core code. Register file importing functions in the same main.py entrypoint as the data engine. Remove options relating to different entrypoints and code directories. The only remaining plugin-specific option in NSandbox is the import directory/mount, i.e. where files to be parsed are placed. Test Plan: this Reviewers: paulfitz Reviewed By: paulfitz Subscribers: dsagal Differential Revision: https://phab.getgrist.com/D2965 2021-08-09 14:51:43 +00:00			`from imports.import_xls import import_file`
(core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows Summary: Use openpyxl instead of messytables (which used xlrd internally) in import_xls.py. Skip empty rows since excel files can easily contain huge numbers of them. Drop support for xls files (which openpyxl doesn't support) in favour of the newer xlsx format. Fix some details relating to python virtualenvs and dependencies, as Jenkins was failing to find new Python dependencies. Test Plan: Mostly relying on existing tests. Updated various tests which referred to xls files instead of xlsx. Added a Python test for skipping empty rows. Reviewers: georgegevoian Reviewed By: georgegevoian Differential Revision: https://phab.getgrist.com/D3406 2022-05-06 15:26:45 +00:00			`return import_file(file_source)`
(core) Move file import plugins into core/sandbox/grist Summary: Move all the plugins python code into the main folder with the core code. Register file importing functions in the same main.py entrypoint as the data engine. Remove options relating to different entrypoints and code directories. The only remaining plugin-specific option in NSandbox is the import directory/mount, i.e. where files to be parsed are placed. Test Plan: this Reviewers: paulfitz Reviewed By: paulfitz Subscribers: dsagal Differential Revision: https://phab.getgrist.com/D2965 2021-08-09 14:51:43 +00:00
			`sandbox.register("xls_parser.parseFile", parse_excel)`

			`def parse_json(file_source, parse_options):`
			`from imports.import_json import parse_file`
			`return parse_file(file_source, parse_options)`

			`sandbox.register("json_parser.parseFile", parse_json)`