(core) revamp snapshot inventory

Summary:
Deliberate changes:
 * save snapshots to s3 prior to migrations.
 * label migration snapshots in s3 metadata.
 * avoid pruning migration snapshots for a month.

Opportunistic changes:
 * Associate document timezone with snapshots, so pruning can respect timezones.
 * Associate actionHash/Num with snapshots.
 * Record time of last change in snapshots (rather than just s3 upload time, which could be a while later).

This ended up being a biggish change, because there was nowhere ideal to put tags (list of possibilities in diff).

Test Plan: added tests

Reviewers: dsagal

Reviewed By: dsagal

Differential Revision: https://phab.getgrist.com/D2646
This commit is contained in:
Paul Fitzpatrick
2020-10-30 12:53:23 -04:00
parent ce824aad34
commit 71519d9e5c
20 changed files with 699 additions and 246 deletions

View File

@@ -44,12 +44,15 @@ export class HostedMetadataManager {
}
/**
* Schedule a call to _update some time from now.
* Schedule a call to _update some time from now. When the update is made, it will
* store the given timestamp in the updated_at column of the docs table for the
* specified document. Timestamp should be an ISO 8601 format time, in UTC, e.g.
* the output of new Date().toISOString()
*/
public scheduleUpdate(docId: string): void {
public scheduleUpdate(docId: string, timestamp: string): void {
// Update updatedAt even if an update is already scheduled - if the update has not yet occurred,
// the more recent updatedAt time will be used.
this._updatedAt[docId] = new Date().toISOString();
this._updatedAt[docId] = timestamp;
if (this._timeout || this._closing) { return; }
const minDelay = this._minPushDelay * 1000;
// Set the push to occur at least the minDelay after the last push time.