(core) add initial support for special shares

Summary:
This gives a mechanism for controlling access control within a document that is distinct from (though implemented with the same machinery as) granular access rules.

It was hard to find a good way to insert this that didn't dissolve in a soup of complications, so here's what I went with:
 * When reading rules, if there are shares, extra rules are added.
 * If there are shares, all rules are made conditional on a "ShareRef" user property.
 * "ShareRef" is null when a doc is accessed in normal way, and the row id of a share when accessed via a share.

There's no UI for controlling shares (George is working on it for forms), but you can do it by editing a `_grist_Shares` table in a document. Suppose you make a fresh document with a single page/table/widget, then to create an empty share you can do:

```
gristDocPageModel.gristDoc.get().docData.sendAction(['AddRecord', '_grist_Shares', null, {linkId: 'xyz', options: '{"publish": true}'}])
```

If you look at the home db now there should be something in the `shares` table:

```
$ sqlite3 -table landing.db "select * from shares"
+----+------------------------+------------------------+--------------+---------+
| id |          key           |         doc_id         |   link_id    | options |
+----+------------------------+------------------------+--------------+---------+
| 1  | gSL4g38PsyautLHnjmXh2K | 4qYuace1xP2CTcPunFdtan | xyz | ...      |
+----+------------------------+------------------------+--------------+---------+
```

If you take the key from that (gSL4g38PsyautLHnjmXh2K in this case) and replace the document's urlId in its URL with `s.<key>` (in this case `s.gSL4g38PsyautLHnjmXh2K` then you can use the regular document landing page (it will be quite blank initially) or API endpoint via the share.

E.g. for me `http://localhost:8080/o/docs/s0gSL4g38PsyautLHnjmXh2K/share-inter-3` accesses the doc.

To actually share some material - useful commands:

```
gristDocPageModel.gristDoc.get().docData.getMetaTable('_grist_Views_section').getRecords()
gristDocPageModel.gristDoc.get().docData.sendAction(['UpdateRecord', '_grist_Views_section', 1, {shareOptions: '{"publish": true, "form": true}'}])
gristDocPageModel.gristDoc.get().docData.getMetaTable('_grist_Pages').getRecords()
gristDocPageModel.gristDoc.get().docData.sendAction(['UpdateRecord', '_grist_Pages', 1, {shareRef: 1}])
```

For a share to be effective, at least one page needs to have its shareRef set to the rowId of the share, and at least one widget on one of those pages needs to have its shareOptions set to {"publish": "true", "form": "true"} (meaning turn on sharing, and include form sharing), and the share itself needs {"publish": true} on its options.

I think special shares are kind of incompatible with public sharing, since by their nature (allowing access to all endpoints) they easily expose the docId, and changing that would be hard.

Test Plan: tests added

Reviewers: dsagal, georgegevoian

Reviewed By: dsagal, georgegevoian

Subscribers: jarek, dsagal

Differential Revision: https://phab.getgrist.com/D4144

app/gen-server/entity/Document.ts

@@ -40,6 +40,12 @@ export class Document extends Resource {
   // fetching user has on the doc, i.e. 'owners', 'editors', 'viewers'
   public access: Role|null;
 
+  // Property that may be returned when the doc is fetched to indicate the share it
+  // is being accessed with. The identifier used is the linkId, which is the share
+  // identifier that is the same between the home database and the document.
+  // The linkId is not a secret, and need only be unique within a document.
+  public linkId?: string|null;
+
   // Property set for forks, containing access the fetching user has on the trunk.
   public trunkAccess?: Role|null;
 

app/gen-server/entity/Share.ts

@@ -0,0 +1,48 @@
+import {ShareOptions} from 'app/common/ShareOptions';
+import {Document} from 'app/gen-server/entity/Document';
+import {nativeValues} from 'app/gen-server/lib/values';
+import {BaseEntity, Column, Entity, JoinColumn, ManyToOne,
+        PrimaryColumn} from 'typeorm';
+
+@Entity({name: 'shares'})
+export class Share extends BaseEntity {
+  /**
+   * A simple integer auto-incrementing identifier for a share.
+   * Suitable for use in within-database references.
+   */
+  @PrimaryColumn({name: 'id', type: Number})
+  public id: number;
+
+  /**
+   * A long string secret to identify the share. Suitable for URLs.
+   * Unique across the database / installation.
+   */
+  @Column({name: 'key', type: String})
+  public key: string;
+
+  /**
+   * A string to identify the share. This identifier is common to the home
+   * database and the document specified by docId. It need only be unique
+   * within that document, and is not a secret. These two properties are
+   * important when you imagine handling documents that are transferred
+   * between installations, or copied, etc.
+   */
+  @Column({name: 'link_id', type: String})
+  public linkId: string;
+
+  /**
+   * The document to which the share belongs.
+   */
+  @Column({name: 'doc_id', type: String})
+  public docId: string;
+
+  /**
+   * Any overall qualifiers on the share.
+   */
+  @Column({name: 'options', type: nativeValues.jsonEntityType})
+  public options: ShareOptions;
+
+  @ManyToOne(type => Document)
+  @JoinColumn({name: 'doc_id'})
+  public doc: Document;
+}

app/gen-server/lib/DocApiForwarder.ts

@@ -3,6 +3,7 @@ import fetch, { RequestInit } from 'node-fetch';
 import {AbortController} from 'node-abort-controller';
 
 import { ApiError } from 'app/common/ApiError';
+import { SHARE_KEY_PREFIX } from 'app/common/gristUrls';
 import { removeTrailingSlash } from 'app/common/gutil';
 import { HomeDBManager } from "app/gen-server/lib/HomeDBManager";
 import { assertAccess, getOrSetDocAuth, getTransitiveHeaders, RequestWithLogin } from 'app/server/lib/Authorizer';
@@ -33,6 +34,13 @@ export class DocApiForwarder {
   }
 
   public addEndpoints(app: express.Application) {
+    app.use((req, res, next) => {
+      if (req.url.startsWith('/api/s/')) {
+        req.url = req.url.replace('/api/s/', `/api/docs/${SHARE_KEY_PREFIX}`);
+      }
+      next();
+    });
+
     // Middleware to forward a request about an existing document that user has access to.
     // We do not check whether the document has been soft-deleted; that will be checked by
     // the worker if needed.

app/gen-server/lib/HomeDBManager.ts

@@ -36,6 +36,7 @@ import {AccessOption, AccessOptionWithRole, Organization} from "app/gen-server/e
 import {Pref} from "app/gen-server/entity/Pref";
 import {getDefaultProductNames, personalFreeFeatures, Product} from "app/gen-server/entity/Product";
 import {Secret} from "app/gen-server/entity/Secret";
+import {Share} from "app/gen-server/entity/Share";
 import {User} from "app/gen-server/entity/User";
 import {Workspace} from "app/gen-server/entity/Workspace";
 import {Limit} from 'app/gen-server/entity/Limit';
@@ -1194,10 +1195,41 @@ export class HomeDBManager extends EventEmitter {
     // Doc permissions of forks are based on the "trunk" document, so make sure
     // we look up permissions of trunk if we are on a fork (we'll fix the permissions
     // up for the fork immediately afterwards).
-    const {trunkId, forkId, forkUserId, snapshotId} = parseUrlId(key.urlId);
+    const {trunkId, forkId, forkUserId, snapshotId,
+           shareKey} = parseUrlId(key.urlId);
+    let doc: Document;
+    if (shareKey) {
+      const res = await (transaction || this._connection).createQueryBuilder()
+        .select('shares')
+        .from(Share, 'shares')
+        .leftJoinAndSelect('shares.doc', 'doc')
+        .where('key = :key', {key: shareKey})
+        .getOne();
+      if (!res) {
+        throw new ApiError('Share not known', 404);
+      }
+      doc = {
+        name: res.doc?.name,
+        id: res.docId,
+        linkId: res.linkId,
+        createdAt: new Date().toISOString(),
+        updatedAt: new Date().toISOString(),
+        isPinned: false,
+        urlId: key.urlId,
+        // For the moment, I don't include a useful workspace.
+        // TODO: look up the document properly, perhaps delegating
+        // to the regular path through this method.
+        workspace: this.unwrapQueryResult<Workspace>(
+          await this.getWorkspace({userId: this.getSupportUserId()},
+                                  this._exampleWorkspaceId)),
+        aliases: [],
+        access: 'editors',  // a share may have view/edit access,
+                            // need to check at granular level
+      } as any;
+      return doc;
+    }
     const urlId = trunkId;
     if (forkId || snapshotId) { key = {...key, urlId}; }
-    let doc: Document;
     if (urlId === NEW_DOCUMENT_CODE) {
       if (!forkId) { throw new ApiError('invalid document identifier', 400); }
       // We imagine current user owning trunk if there is no embedded userId, or
@@ -3022,6 +3054,41 @@ export class HomeDBManager extends EventEmitter {
     return limitOrError;
   }
 
+  public async syncShares(docId: string, shares: ShareInfo[]) {
+    return this._connection.transaction(async manager => {
+      for (const share of shares) {
+        const key = makeId();
+        await manager.createQueryBuilder()
+          .insert()
+        // if urlId has been used before, update it
+          .onConflict(`(doc_id, link_id) DO UPDATE SET options = :options`)
+          .setParameter('options', share.options)
+          .into(Share)
+          .values({
+            linkId: share.linkId,
+            docId,
+            options: JSON.parse(share.options),
+            key,
+          })
+          .execute();
+      }
+      const dbShares = await manager.createQueryBuilder()
+        .select('shares')
+        .from(Share, 'shares')
+        .where('doc_id = :docId', {docId})
+        .getMany();
+      const activeLinkIds = new Set(shares.map(share => share.linkId));
+      const oldShares = dbShares.filter(share => !activeLinkIds.has(share.linkId));
+      if (oldShares.length > 0) {
+        await manager.createQueryBuilder()
+          .delete()
+          .from('shares')
+          .whereInIds(oldShares.map(share => share.id))
+          .execute();
+      }
+    });
+  }
+
   private async _getOrCreateLimit(accountId: number, limitType: LimitType, force: boolean): Promise<Limit|null> {
     if (accountId === 0) {
       throw new Error(`getLimit: called for not existing account`);
@@ -4866,3 +4933,8 @@ export function getDocAuthKeyFromScope(scope: Scope): DocAuthKey {
   if (!urlId) { throw new Error('document required'); }
   return {urlId, userId, org};
 }
+
+interface ShareInfo {
+  linkId: string;
+  options: string;
+}

app/gen-server/migration/1701557445716-Shares.ts

@@ -0,0 +1,54 @@
+import { nativeValues } from 'app/gen-server/lib/values';
+import { MigrationInterface, QueryRunner, Table, TableForeignKey, TableUnique } from 'typeorm';
+
+export class Shares1701557445716 implements MigrationInterface {
+
+  public async up(queryRunner: QueryRunner): Promise<void> {
+    await queryRunner.createTable(new Table({
+      name: 'shares',
+      columns: [
+        {
+          name: 'id',
+          type: 'integer',
+          isGenerated: true,
+          generationStrategy: 'increment',
+          isPrimary: true
+        },
+        {
+          name: 'key',
+          type: 'varchar',
+          isUnique: true,
+        },
+        {
+          name: 'doc_id',
+          type: 'varchar',
+        },
+        {
+          name: 'link_id',
+          type: 'varchar',
+        },
+        {
+          name: 'options',
+          type: nativeValues.jsonType,
+        },
+      ]
+    }));
+    await queryRunner.createForeignKeys('shares', [
+      new TableForeignKey({
+        columnNames: ['doc_id'],
+        referencedTableName: 'docs',
+        referencedColumnNames: ['id'],
+        onDelete: 'CASCADE', // delete share if doc goes away
+      }),
+    ]);
+    await queryRunner.createUniqueConstraints('shares', [
+      new TableUnique({
+        columnNames: ['doc_id', 'link_id'],
+      }),
+    ]);
+  }
+
+  public async down(queryRunner: QueryRunner): Promise<void> {
+    await queryRunner.dropTable('shares');
+  }
+}