(core) add machinery for self-managed flavor of Grist

Summary:
Currently, we have two ways that we deliver Grist. One is grist-core,
which has simple defaults and is relatively easy for third parties to
deploy. The second is our internal build for our SaaS, which is the
opposite. For self-managed Grist, a planned paid on-premise version
of Grist, I adopt the following approach:

 * Use the `grist-core` build mechanism, extending it to accept an
   overlay of extra code if present.
 * Extra code is supplied in a self-contained `ext` directory, with
   an `ext/app` directory that is of same structure as core `app`
   and `stubs/app`.
 * The `ext` directory also contains information about extra
   node dependencies needed beyond that of `grist-core`.
 * The `ext` directory is contained within our monorepo rather than
   `grist-core` since it may contain material not under the Apache
   license.

Docker builds are achieved in our monorepo by using the `--build-context`
functionality to add in `ext` during the regular `grist-core` build:

```
docker buildx build --load -t gristlabs/grist-ee --build-context=ext=../ext .
```

Incremental builds in our monorepo are achieved with the `build_core.sh` helper,
like:

```
buildtools/build_core.sh /tmp/self-managed
cd /tmp/self-managed
yarn start
```

The initial `ext` directory contains material for snapshotting to S3.
If you build the docker image as above, and have S3 access, you can
do something like:

```
docker run -p 8484:8484 --env GRIST_SESSION_SECRET=a-secret \
  --env GRIST_DOCS_S3_BUCKET=grist-docs-test \
  --env GRIST_DOCS_S3_PREFIX=self-managed \
  -v $HOME/.aws:/root/.aws -it gristlabs/grist-ee
```

This will start a version of Grist that is like `grist-core` but with
S3 snapshots enabled. To release this code to `grist-core`, it would
just need to move from `ext/app` to `app` within core.

I tried a lot of ways of organizing self-managed Grist, and this was
what made me happiest. There are a lot of trade-offs, but here is what
I was looking for:

 * Only OSS-code in grist-core. Adding mixed-license material there
   feels unfair to people already working with the repo. That said,
   a possible future is to move away from our private monorepo to
   a public mixed-licence repo, which could have the same relationship
   with grist-core as the monorepo has.
 * Minimal differences between self-managed builds and one of our
   existing builds, ideally hewing as close to grist-core as possible
   for ease of documentation, debugging, and maintenance.
 * Ideally, docker builds without copying files around (the new
   `--build-context` functionality made that possible).
 * Compatibility with monorepo build.

Expressing dependencies of the extra code in `ext` proved tricky to
do in a clean way. Yarn/npm fought me every step of the way - everything
related to optional dependencies was unsatisfactory in some respect.
Yarn2 is flexible but smells like it might be overreach. In the end,
organizing to install non-core dependencies one directory up from the
main build was a good simple trick that saved my bacon.

This diff gets us to the point of building `grist-ee` images conveniently,
but there isn't a public repo people can go look at to see its source. This
could be generated by taking `grist-core`, adding the `ext` directory
to it, and pushing to a distinct repository. I'm not in a hurry to do that,
since a PR to that repo would be hard to sync with our monorepo and
`grist-core`. Also, we don't have any licensing text ready for the `ext`
directory. So leaving that for future work.

Test Plan: manual

Reviewers: georgegevoian, alexmojaki

Reviewed By: georgegevoian, alexmojaki

Differential Revision: https://phab.getgrist.com/D3415

app/server/lib/ICreate.ts

@@ -8,6 +8,7 @@ import {IBilling} from 'app/server/lib/IBilling';
 import {INotifier} from 'app/server/lib/INotifier';
 import {ISandbox, ISandboxCreationOptions} from 'app/server/lib/ISandbox';
 import {IShell} from 'app/server/lib/IShell';
+import {createSandbox} from 'app/server/lib/NSandbox';
 
 export interface ICreate {
 
@@ -20,14 +21,7 @@ export interface ICreate {
   //  - meta. This store need not be versioned, and can be eventually consistent.
   // For test purposes an extra prefix may be supplied.  Stores with different prefixes
   // should not interfere with each other.
-  // innerCreate should be a function returning the core ExternalStorage implementation,
-  // which this method may wrap in additional layer(s) of ExternalStorage.
-  // Uses S3 by default in hosted Grist.
-  ExternalStorage(
-    purpose: 'doc' | 'meta',
-    testExtraPrefix: string,
-    innerCreate?: (bucket: string) => ExternalStorage
-  ): ExternalStorage | undefined;
+  ExternalStorage(purpose: 'doc' | 'meta', testExtraPrefix: string): ExternalStorage | undefined;
 
   ActiveDoc(docManager: DocManager, docName: string, options: ICreateActiveDocOptions): ActiveDoc;
   NSandbox(options: ISandboxCreationOptions): ISandbox;
@@ -42,3 +36,60 @@ export interface ICreateActiveDocOptions {
   docUrl?: string;
   doc?: Document;
 }
+
+export interface ICreateStorageOptions {
+  check(): Record<string, string>|undefined;
+  create(purpose: 'doc'|'meta', extraPrefix: string): ExternalStorage|undefined;
+}
+
+export function makeSimpleCreator(opts: {
+  sessionSecret?: string,
+  storage?: ICreateStorageOptions[],
+}): ICreate {
+  return {
+    Billing() {
+      return {
+        addEndpoints() { /* do nothing */ },
+        addEventHandlers() { /* do nothing */ },
+        addWebhooks() { /* do nothing */ }
+      };
+    },
+    Notifier() {
+      return {
+        get testPending() { return false; },
+        deleteUser()      { throw new Error('deleteUser unavailable'); },
+      };
+    },
+    Shell() {
+      return {
+        moveItemToTrash()  { throw new Error('moveToTrash unavailable'); },
+        showItemInFolder() { throw new Error('showItemInFolder unavailable'); }
+      };
+    },
+    ExternalStorage(purpose, extraPrefix) {
+      for (const storage of opts.storage || []) {
+        const config = storage.check();
+        if (config) { return storage.create(purpose, extraPrefix); }
+      }
+      return undefined;
+    },
+    ActiveDoc(docManager, docName, options) { return new ActiveDoc(docManager, docName, options); },
+    NSandbox(options) {
+      return createSandbox('unsandboxed', options);
+    },
+    sessionSecret() {
+      const secret = process.env.GRIST_SESSION_SECRET || opts.sessionSecret;
+      if (!secret) {
+        throw new Error('need GRIST_SESSION_SECRET');
+      }
+      return secret;
+    },
+    configurationOptions() {
+      for (const storage of opts.storage || []) {
+        const config = storage.check();
+        if (config) { return config; }
+      }
+      return {};
+    }
+  };
+}