Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2024-10-27 20:44:07 +00:00
Code Issues Projects Releases Wiki Activity
b878395c21
gristlabs_grist-core/app/server/lib
History
Alex Hall 6c90de4d62 (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 
Summary:
Use openpyxl instead of messytables (which used xlrd internally) in import_xls.py.

Skip empty rows since excel files can easily contain huge numbers of them.

Drop support for xls files (which openpyxl doesn't support) in favour of the newer xlsx format.

Fix some details relating to python virtualenvs and dependencies, as Jenkins was failing to find new Python dependencies.

Test Plan: Mostly relying on existing tests. Updated various tests which referred to xls files instead of xlsx. Added a Python test for skipping empty rows.

Reviewers: georgegevoian

Reviewed By: georgegevoian

Differential Revision: https://phab.getgrist.com/D3406
2022-05-12 14:43:21 +02:00
..
ACLFormula.ts (core) add OWNERS='owners', EDITOR='editors', VIEWER='viewers' to condition formulas 2021-03-19 18:20:33 -04:00
ActionHistory.ts (core) do not look at content of recent actions when loading documents 2021-09-29 11:27:02 -04:00
ActionHistoryImpl.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
ActionSummary.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
ActiveDoc.ts (core) Add attachment and data size usage 2022-05-04 13:46:55 -07:00
ActiveDocImport.ts (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
AppEndpoint.ts (core) make Grist easier to run with a single server 2022-03-05 13:30:45 -05:00
Authorizer.ts (core) when redirecting, use protocol in APP_HOME_URL if available 2022-04-28 09:13:27 -04:00
BrowserSession.ts (core) Add authSubject and authProvider to sessions 2022-04-11 11:42:02 -07:00
checksumFile.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
Client.ts (core) get user.Name through same mechanism as user.id for websocket Client 2022-04-14 12:49:35 -04:00
Comm.js (core) get user.Name through same mechanism as user.id for websocket Client 2022-04-14 12:49:35 -04:00
dbUtils.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
DiscourseConnect.ts (core) when redirecting, use protocol in APP_HOME_URL if available 2022-04-28 09:13:27 -04:00
DocApi.ts (core) Improving custom widget API. Changing destroy function signature. 2022-05-05 16:34:26 +02:00
DocClients.ts (core) Add LogMethods helper and use it for more JSON data in logs. Reduce unhelpful logging. 2021-10-25 10:25:18 -04:00
DocManager.ts (core) Add attachment and data size usage 2022-05-04 13:46:55 -07:00
DocPluginData.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
DocPluginManager.ts (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
DocSession.ts (core) add a user.SessionID value for trigger formulas and granular access rules 2022-02-22 12:50:43 -05:00
DocSnapshots.ts (core) Store monthly snapshots for 8 years to give Enterprise plans a more significant advantage 2022-04-05 18:11:13 +02:00
DocStorage.ts (core) Add attachment and data size usage 2022-05-04 13:46:55 -07:00
DocStorageManager.ts (core) uncheck FullCopy special when copying/forking a document 2021-04-29 08:56:54 -04:00
docUtils.d.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
docUtils.js Correct spelling mistakes 2022-02-19 09:46:49 +00:00
DocWorker.ts (core) Clean up and refactor uses of HomeDBManager.getDoc 2022-03-24 13:42:36 +02:00
DocWorkerMap.ts (core) Add flexibility to daily API usage limit 2022-04-28 16:22:18 +02:00
ExcelFormatter.ts (core) Custom Widget column mapping feature. 2022-02-08 17:41:04 +01:00
ExpandedQuery.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
Export.ts (core) Use MetaTableData more 2021-12-07 17:09:58 +02:00
ExportCSV.ts (core) Use MetaTableData more 2021-12-07 17:09:58 +02:00
ExportXLSX.ts (core) Exposing more descriptive errors from exports 2021-11-30 17:26:32 +01:00
expressWrap.ts (core) Include altSessionId in logs 2022-04-08 16:40:34 -07:00
ExternalStorage.ts (core) Remove expired attachments every hour and on shutdown 2022-04-22 20:43:59 +02:00
extractOrg.ts (core) when redirecting, use protocol in APP_HOME_URL if available 2022-04-28 09:13:27 -04:00
FileParserElement.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
filterUtils.ts (core) uncheck FullCopy special when copying/forking a document 2021-04-29 08:56:54 -04:00
FlexServer.ts (core) when redirecting, use protocol in APP_HOME_URL if available 2022-04-28 09:13:27 -04:00
ForwardAuthLogin.ts (core) add a login method based on headers 2022-04-04 18:36:09 -04:00
GoogleAuth.ts (core) Exposing more descriptive errors from exports 2021-11-30 17:26:32 +01:00
GoogleExport.ts (core) Include altSessionId in logs 2022-04-08 16:40:34 -07:00
GoogleImport.ts (core) Extending Google Drive integration scope 2021-10-01 10:47:12 +02:00
GranularAccess.ts (core) Add attachment and data size usage 2022-05-04 13:46:55 -07:00
GristServer.ts (core) add a login method based on headers 2022-04-04 18:36:09 -04:00
gristSessions.ts (core) Implement DiscourseConnect to enable easy sign-in to community forum 2021-10-01 11:24:22 -04:00
guessExt.ts (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
HashUtil.ts (core) add more detail to /compare endpoint 2020-09-18 16:31:29 -04:00
HostedMetadataManager.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
HostedStorageManager.ts (core) Add AzureExternalStorage 2022-05-09 21:44:57 +02:00
IBilling.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
IChecksumStore.ts (core) revamp snapshot inventory 2020-10-30 13:52:46 -04:00
ICreate.ts (core) Add AzureExternalStorage 2022-05-09 21:44:57 +02:00
IDocStorageManager.ts (core) uncheck FullCopy special when copying/forking a document 2021-04-29 08:56:54 -04:00
idUtils.ts (core) start reconciling forking with granular access 2021-01-12 14:08:49 -05:00
IElectionStore.ts (core) move some material to core that slipped through in a rebase 2020-07-23 11:29:05 -04:00
initialDocSql.ts (core) Distinct style rules for summary columns 2022-04-27 20:51:23 +02:00
INotifier.ts (core) add a tool for deleting a user 2021-09-29 12:08:23 -04:00
ISandbox.ts (core) New type conversion in the backend 2022-02-04 20:28:13 +02:00
IShell.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
ITestingHooks-ti.ts (core) be careful when reassigning a doc to a worker it was on before 2022-03-08 17:20:01 -05:00
ITestingHooks.ts (core) be careful when reassigning a doc to a worker it was on before 2022-03-08 17:20:01 -05:00
log.ts (core) give instructions on using Grist with docker 2020-10-28 13:59:13 -04:00
LogMethods.ts (core) Add LogMethods helper and use it for more JSON data in logs. Reduce unhelpful logging. 2021-10-25 10:25:18 -04:00
manifest.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
MinimalLogin.ts (core) Add authSubject and authProvider to sessions 2022-04-11 11:42:02 -07:00
NSandbox.ts (core) Switch excel import parsing from messytables+xlrd to openpyxl, and ignore empty rows 2022-05-12 14:43:21 +02:00
OnDemandActions.ts (core) Use MetaTableData more 2021-12-07 17:09:58 +02:00
PermissionInfo.ts (core) Add 'user' variable to trigger formulas 2021-07-15 15:18:32 -07:00
Permit.ts (core) Add new Grist login page 2022-04-01 15:24:19 -07:00
places.ts (core) fix docker packaging after core shuffle 2020-07-22 14:45:42 -04:00
PluginEndpoint.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
PluginManager.ts Correct spelling mistakes 2022-02-19 09:46:49 +00:00
reportTimeTaken.ts (core) Log the time taken by decodeActionFromRow() operations. 2021-08-20 11:28:33 -04:00
requestUtils.ts (core) when redirecting, use protocol in APP_HOME_URL if available 2022-04-28 09:13:27 -04:00
RowAccess.ts (core) Add rules to eslint to better match our coding conventions. 2021-05-24 12:56:18 -04:00
SafePythonComponent.ts (core) Move file import plugins into core/sandbox/grist 2021-08-09 18:37:14 +02:00
SamlConfig.ts (core) when redirecting, use protocol in APP_HOME_URL if available 2022-04-28 09:13:27 -04:00
SandboxControl.ts (core) freshen tests for python3 2021-11-10 10:46:12 -05:00
sandboxUtil.js (core) move home server into core 2020-07-21 20:39:10 -04:00
sendAppPage.ts (core) Record new user sign-ups 2022-03-12 14:34:46 -08:00
ServerColumnGetters.ts (core) Adding sort options for columns. 2021-11-03 15:31:39 +01:00
ServerLocale.ts (core) Simple localization support and currency selector. 2021-08-26 13:36:49 -07:00
serverUtils.ts (core) support setting python version of new docs with PYTHON_VERSION_ON_CREATION 2021-11-05 10:51:18 -04:00
Sessions.ts (core) make Grist easier to run with a single server 2022-03-05 13:30:45 -05:00
Sharing.ts (core) Add attachment and data size usage 2022-05-04 13:46:55 -07:00
shortDesc.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
shutdown.js (core) move home server into core 2020-07-21 20:39:10 -04:00
SQLiteDB.ts (core) Remove expired attachments every hour and on shutdown 2022-04-22 20:43:59 +02:00
TableMetadataLoader.ts (core) open documents without blocking on data engine 2021-10-01 10:18:56 -04:00
TagChecker.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
TestingHooks.ts (core) be careful when reassigning a doc to a worker it was on before 2022-03-08 17:20:01 -05:00
TestLogin.ts (core) Add new Grist login page 2022-04-01 15:24:19 -07:00
Throttle.ts (core) tweak throttling to work for gvisor/runsc 2021-11-04 17:23:43 -04:00
TimeQuery.ts (core) Raw renames 2022-04-27 22:21:55 +02:00
Triggers.ts (core) Remove expired attachments every hour and on shutdown 2022-04-22 20:43:59 +02:00
UnsafeNodeComponent.ts (core) move home server into core 2020-07-21 20:39:10 -04:00
uploads.ts (core) Include altSessionId in logs 2022-04-08 16:40:34 -07:00
WidgetRepository.ts (core) Widget options api 2022-01-13 11:10:17 +01:00
WorkCoordinator.ts (core) move home server into core 2020-07-21 20:39:10 -04:00