Archives/gristlabs_grist-core
1
0
Fork 0
mirror of https://github.com/gristlabs/grist-core.git synced 2026-03-02 04:09:24 +00:00
Code Issues Projects Releases Wiki Activity

(core) Remove DB transaction from webhook update, add mutex to all webhook endpoints

Browse Source 
Summary:
This removes problematic code that was holding a HomeDB transaction while applying user actions which could hang indefinitely, especially if the webhook queue is full as in https://grist.slack.com/archives/C05DBJ6LA1F/p1698159750945949.

The discussion about adding this code is here: https://phab.getgrist.com/D3821#inline-45054

The initial motivation was to roll back HomeDB changes if something went wrong while applying user actions, to avoid saving only part of the changes the user requested. I think it's actually fine to just allow such a partial save to happen - I don't see anything particularly undesirable about keeping an update to the webhook URL if other updates requested by the user didn't also get applied, as the fields don't affect each other.

The comment approving the transaction approach said "so we shouldn't end up leave the transaction hanging around too long" which has been falsified.

It looks like there was also some desire to prevent a mess caused by multiple simultaneous calls to this endpoint, which the transaction may have helped with a little, but didn't really seem like a solution. Comments in `Triggers.ts` also mention fears of race conditions when clearing (some of) the queue and the need for some locking. So I wrapped all webhook-related endpoints in a simple `Mutex` held by the `ActiveDoc` to prevent simultaneous changes. I *think* this is a good thing. These endpoints shouldn't be called frequently enough to create a performance issue, and this shouldn't affect actually sending webhook events when records are added/updated. And it does seem like interleaving calls to these endpoints could cause very weird problems.

Test Plan: Nothing yet, I'd like to hear if others think this is sensible.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D4111
This commit is contained in:
Alex Hall
2023-11-14 15:46:33 +02:00
parent dda1b5cf1b
commit 3dfe4be5f3
3 changed files with 51 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
app/common/gutil.ts
View File

@@ -886,7 +886,9 @@ export function isValidHex(val: string): boolean {
* Resolves to true if promise is still pending after msec milliseconds have passed. Otherwise
* returns false, including when promise is rejected.
*/
export async function timeoutReached(msec: number, promise: Promise<unknown>): Promise<boolean> {
export async function timeoutReached(
msec: number, promise: Promise<unknown>, options: {rethrow: boolean} = {rethrow: false}
): Promise<boolean> {
const timedOut = {};
// Be careful to clean up the timer after ourselves, so it doesn't remain in the event loop.
let timer: NodeJS.Timer;
@@ -895,6 +897,9 @@ export async function timeoutReached(msec: number, promise: Promise<unknown>): P
const res = await Promise.race([promise, delayPromise]);
return res == timedOut;
} catch (err) {
if (options.rethrow) {
throw err;
}
return false;
} finally {
clearTimeout(timer!);