(core) Webhook event queue on redis

Summary:
Push webhook events to redis queue with key based on docId.

Remove events from redis after sending using LTRIM.

Put failed events back on the end of the queue under normal circumstances.

When the event queue gets too long:
- Wait until it gets consumed before continuing.
- Drop failed events (i.e. don't put them back on the end of the queue)
- Limit webhook retries to 5

Test Plan: Tested that interactions with redis are as expected using redis MONITOR command.

Reviewers: paulfitz

Reviewed By: paulfitz

Differential Revision: https://phab.getgrist.com/D3100
This commit is contained in:
Alex Hall
2021-11-03 21:09:27 +02:00
parent 1db138d7ac
commit 4a70283292
2 changed files with 137 additions and 25 deletions

View File

@@ -52,6 +52,8 @@ declare module "redis" {
public ttlAsync(key: string): Promise<number|null>;
public unwatchAsync(): Promise<'OK'>;
public watchAsync(key: string): Promise<void>;
public lrangeAsync(key: string, start: number, end: number): Promise<string[]>;
public rpushAsync(key: string, ...vals: string[]): Promise<number>;
}
class Multi {
@@ -67,5 +69,7 @@ declare module "redis" {
public smembers(key: string): Multi;
public srandmember(key: string): Multi;
public srem(key: string, val: string): Multi;
public rpush(key: string, ...vals: string[]): Multi;
public ltrim(key: string, start: number, end: number): Multi;
}
}