6 Commits (6f00106d7c26802964b115abec8dcc472fc171a1)

Author	SHA1	Message	Date
Alex Hall	2c9ae6dc94	(core) Enforce daily limit on API usage ... Summary: Keep track of the number of API requests made for this document today in redis. Uses local caches of the count and the document so that usually requests can proceed without waiting for redis or the database. Moved the free standing function apiThrottle to become a method to avoid adding another layer of request handler callbacks. Test Plan: Added a DocApi test Reviewers: paulfitz Reviewed By: paulfitz Subscribers: dsagal Differential Revision: https://phab.getgrist.com/D3327	3 years ago
Edward Betts	d6e0e1fee3	Correct spelling mistakes	3 years ago
Paul Fitzpatrick	54beaede84	(core) revive saml support and test against Auth0 ... Summary: SAML support had broken due to SameSite changes in browsers. This makes it work again, and tests it against Auth0 (now owned by Okta). Logging in and out works. The logged out state is confusing, and may not be complete. The "Add Account" menu item doesn't work. But with this, an important part of self-hosting becomes easier. SAML support works also in grist-core, for site pages, but there is a glitch on document pages that I'll look into separately. Test Plan: tested manually Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2976	3 years ago
Paul Fitzpatrick	d6ff1361cb	(core) support GRIST_WORKER_GROUP to place worker into an exclusive group ... Summary: In an emergency, we may want to serve certain documents with "old" workers as we fix problems. This diff adds some support for that. * Creates duplicate task definitions and services for staging and production doc workers (called grist-docs-staging2 and grist-docs-prod2), pulling from distinct docker tags (staging2 and prod2). The services are set to have zero workers until we need them. * These new workers are started with a new env variable `GRIST_WORKER_GROUP` set to `secondary`. * The `GRIST_WORKER_GROUP` variable, if set, makes the worker available to documents in the named group, and only that group. * An unauthenticated `/assign` endpoint is added to documents which, when POSTed to, checks that the doc is served by a worker in the desired group for that doc (as set manually in redis), and if not frees the doc up for reassignment. This makes it possible to move individual docs between workers without redeployments. The bash scripts added are a record of how the task definitions + services were created. The services could just have been copied manually, but the task definitions will need to be updated whenever the definitions for the main doc workers are updated, so it is worth scripting that. For example, if a certain document were to fail on a new deployment of Grist, but rolling back the full deployment wasn't practical: * Set prod2 tag in docker to desired codebase for that document * Set desired_count for grist-docs-prod2 service to non-zero * Set doc-<docid>-group for that doc in redis to secondary * Hit /api/docs/<docid>/assign to move the doc to grist-docs-prod2 (If the document needs to be reverted to a previous snapshot, that currently would need doing manually - could be made simpler, but not in scope of this diff). Test Plan: added tests Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2649	4 years ago
Paul Fitzpatrick	71519d9e5c	(core) revamp snapshot inventory ... Summary: Deliberate changes: * save snapshots to s3 prior to migrations. * label migration snapshots in s3 metadata. * avoid pruning migration snapshots for a month. Opportunistic changes: * Associate document timezone with snapshots, so pruning can respect timezones. * Associate actionHash/Num with snapshots. * Record time of last change in snapshots (rather than just s3 upload time, which could be a while later). This ended up being a biggish change, because there was nowhere ideal to put tags (list of possibilities in diff). Test Plan: added tests Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2646	4 years ago
Paul Fitzpatrick	5ef889addd	(core) move home server into core ... Summary: This moves enough server material into core to run a home server. The data engine is not yet incorporated (though in manual testing it works when ported). Test Plan: existing tests pass Reviewers: dsagal Reviewed By: dsagal Differential Revision: https://phab.getgrist.com/D2552	4 years ago