From e06f0bc1d868193fc059ac7f3364005f80d525ce Mon Sep 17 00:00:00 2001 From: Alex Hall Date: Sun, 21 Aug 2022 14:51:11 +0200 Subject: [PATCH] (core) Retry flaky daily API usage test Summary: This particular test fails often enough to be annoying but not often enough to be worrying. It's not clear why, but it seems like a race condition involving redis. Fixing the test 'properly' seems hard and not worth the effort. Looking at the past 20 Jenkins builds, I found this test failing once. If we assume that the probability of failing is 1/20 (it's probably less since the test actually runs 3 times with different server configurations) then the probability of failing 3 times independently is (1/20)^3 = 1/8000, so `this.retry(3)` seems like a good enough solution. See also: https://grist.slack.com/archives/C0234CPPXPA/p1652909955773049 Test Plan: Manually made the test fail randomly 90% of the time, in which case `this.retries(3)` was not usually enough to prevent failures, but `this.retries(300)` was. Reviewers: paulfitz Reviewed By: paulfitz Subscribers: paulfitz Differential Revision: https://phab.getgrist.com/D3595 --- test/server/lib/DocApi.ts | 1 + 1 file changed, 1 insertion(+) diff --git a/test/server/lib/DocApi.ts b/test/server/lib/DocApi.ts index 336e8bc3..634fd311 100644 --- a/test/server/lib/DocApi.ts +++ b/test/server/lib/DocApi.ts @@ -2367,6 +2367,7 @@ function testDocApi() { }); it("limits daily API usage and sets the correct keys in redis", async function() { + this.retries(3); // Make a new document in a free team site, currently the only real product which limits daily API usage. const freeTeamApi = makeUserApi('freeteam'); const workspaceId = await getWorkspaceId(freeTeamApi, 'FreeTeamWs');