(core) be careful when reassigning a doc to a worker it was on before

Summary:
Importing a .grist document is implemented in a somewhat clunky way, in a multi-worker setup.

 * First a random worker receives the upload, and updates Grist's various stores appropriately (database, redis, s3).
 * Then a random worker is assigned to serve the document.

If the worker serving the document fails, there is a chance the it will end up assigned to the worker that handled its upload. Currently the worker will misbehave in this case. This diff:

 * Ports a multi-worker test from test/home to run in test/s3, and adds a test simulating a bad scenario seen in the wild.
 * Fixes persistence of any existing document checksum in redis when a worker is assigned.
 * Adds a check when assigned a document to serve, and finding that document already cached locally. It isn't safe to rely only on the document checksum in redis, since that may have expired.
 * Explicitly claims the document on the uploading worker, so this situation becomes even less likely to arise.

Test Plan: added test

Reviewers: dsagal

Reviewed By: dsagal

Subscribers: dsagal

Differential Revision: https://phab.getgrist.com/D3305

test/nbrowser/gristUtils.ts

@@ -151,6 +151,21 @@ export async function testCurrentUrl(pattern: RegExp|string) {
   return (typeof pattern === 'string') ? url.includes(pattern) : pattern.test(url);
 }
 
+export async function getDocWorkerUrls(): Promise<string[]> {
+  const result = await driver.wait(() => driver.executeScript(`
+    return Array.from(window.gristApp.comm.listConnections().values());
+  `), 1000) as string[];
+  return result;
+}
+
+export async function getDocWorkerUrl(): Promise<string> {
+  const urls = await getDocWorkerUrls();
+  if (urls.length > 1) {
+    throw new Error(`Expected a single docWorker URL, received ${urls}`);
+  }
+  return urls[0] || '';
+}
+
 export async function waitForUrl(pattern: RegExp|string, waitMs: number = 2000) {
   await driver.wait(() => testCurrentUrl(pattern), waitMs);
 }