Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2026-03-02 04:09:24 +00:00
Code Issues Projects Releases Wiki Activity

(core) Enable incremental imports

Browse Source 
Summary:
The import dialog now has an option to 'Update existing records',
which when checked will allow for selection of 1 or more fields
to match source and destination tables on.

If all fields match, then the matched record in the
destination table will be merged with the incoming record
from the source table. This means the incoming values will
replace the destination table values, unless the incoming
values are blank.

Additional merge strategies are implemented in the data
engine, but the import dialog only uses one of the
strategies currently. The others can be exposed in the UI
in the future, and tweak the behavior of how source
and destination values should be merged in different contexts,
such as when blank values exist.

Test Plan: Python and browser tests.

Reviewers: paulfitz

Reviewed By: paulfitz

Subscribers: alexmojaki

Differential Revision: https://phab.getgrist.com/D3020
This commit is contained in:
George Gevoian
2021-09-14 23:12:34 -07:00
parent a543e5194a
commit 8a7edb6257
9 changed files with 914 additions and 135 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
app/common/ActiveDocAPI.ts
View File

@@ -56,6 +56,20 @@ export interface ImportTableResult {
destTableId: string|null;
}
export interface MergeStrategy {
type: 'replace-with-nonblank-source' | 'replace-all-fields' | 'replace-blank-fields-only';
}
export interface MergeOptions {
mergeCols: string[]; // Columns to use as merge keys for incremental imports.
mergeStrategy: MergeStrategy; // Determines how matched records should be merged between 2 tables.
}
export interface ImportOptions {
parseOptions?: ParseOptions; // Options for parsing the source file.
mergeOptions?: Array<MergeOptions|null>; // Options for merging fields, indexed by uploadFileIndex.
}
/**
* Represents a query for Grist data. The tableId is required. An empty set of filters indicates
* the full table. Examples:
@@ -159,8 +173,8 @@ export interface ActiveDocAPI {
/**
* Finishes import files, creates the new tables, and cleans up temporary hidden tables and uploads.
*/
finishImportFiles(dataSource: DataSourceTransformed,
parseOptions: ParseOptions, prevTableIds: string[]): Promise<ImportResult>;
finishImportFiles(dataSource: DataSourceTransformed, prevTableIds: string[],
options: ImportOptions): Promise<ImportResult>;
/**
* Cancels import files, cleans up temporary hidden tables and uploads.