You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Observed in CI run #33684 (attempt 1 failed, attempt 2 passed — confirming this is a flaky/timing issue).
The logs also report a worker teardown leak:
A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --detectOpenHandles to find leaks. Active timers can also cause this.
Likely cause
The tests use `waitForNetworkPromises()` / `waitForBatchedUpdates()` before asserting on `global.fetch` call count. A race condition between the `SequentialQueue.unpause()` call and the fetch mock resolution can cause the second fetch to fire after the assertion runs. The worker teardown leak suggests an open handle (timer or promise) that affects subsequent test isolation.
🚨 Edited by proposal-police: This proposal was edited at 2026-05-14 12:46:58 UTC.
Proposal
What is the root cause of that problem?
Both flaky tests push two requests onto the SequentialQueue, then assert expect(global.fetch).toHaveBeenCalledTimes(2) after only a fixed number of microtask/macrotask flushes — waitForNetworkPromises() (two waitForBatchedUpdates rounds) in the first test, and a singlewaitForBatchedUpdates() in the second:
But the queue does not process both requests in a fixed number of cycles. process() handles one request at a time through a recursive chain — it resolves the fetch mock (two async hops for the async mock + its async json()), runs the middleware chain (which performs async Onyx writes), applies the response via Onyx.update, and only then does .then(() => process()) pick up the next request and fire its fetch:
// A response might indicate that the queue should be paused. This happens when a gap in onyx updates is detected between the client and the server and
// that gap needs resolved before the queue can continue.
if(response?.shouldPauseQueue){
Log.info("[SequentialQueue] Handled 'shouldPauseQueue' in response. Pausing the queue.");
pause();
}
Log.info('[SequentialQueue] Removing persisted request because it was processed successfully.',false,{
Log.info('[SequentialQueue] Continuing to process next request');
returnprocess();
The async depth required for the secondfetch to fire is variable. Under CI scheduling pressure it exceeds the fixed number of flushes, so the assertion runs while only the first fetch has been called → toHaveBeenCalledTimes(2) fails. The same unbounded process() chain still running after the test returns is also the open handle behind the "worker process failed to exit gracefully" teardown leak.
Note: commit a7ede572585 improved this file's test isolation (beforeEach resets), but it did not address this assertion-timing race.
What changes do you think we should make in order to solve the problem?
Stop waiting a fixed number of cycles and instead wait for the queue to actually drain, using the existing SequentialQueue.waitForIdle() idiom (already used in ReportAttachments, WorkspaceUpgradeTest, PaginationTest). push() of the first request synchronously calls flush(true), which resets isReadyPromise before the test reaches its wait; that promise resolves in process().finally() only after the whole recursive chain has drained both requests:
(SequentialQueue is already imported in this file; the trailing waitForBatchedUpdates() in the second test keeps the follow-up Onyx reads consistent.) This makes the fetch count deterministic and guarantees the queue is fully drained before teardown, which also removes the worker-exit leak.
The identical fixed-cycle pattern exists at the other SequentialQueue.unpause() sites in this file (L93-L94, L198-L199, L253-L254, L438-L439); applying the same waitForIdle() wait there prevents the same flake from resurfacing in those tests.
What alternative solutions did you explore? (Optional)
Adding more waitForBatchedUpdates() cycles (or widening waitForNetworkPromises) — rejected: it only lowers the probability of the race, it does not eliminate it, since the queue's async depth is unbounded relative to any fixed cycle count. await SequentialQueue.getCurrentRequest() — rejected: currentRequestPromise is null synchronously after unpause() (the chain starts inside an async Onyx.connectWithoutView callback), so awaiting it immediately resolves and waits for nothing.
The two failing tests wait a fixed number of async cycles before asserting on global.fetch call count, but SequentialQueue requires a variable number of async hops between processing request #1 and issuing fetch #2.
Between the two fetches, the queue chains through: fetch() → response.json() → middleware processing (HandleUnusedOptimisticID + SaveResponseInOnyx with Onyx writes) → endPersistedRequestAndRemoveFromQueue (another Onyx write) → recursive process() → fetch #2. That's 5-6+ async hops.
Test 1 (MiddlewareTest.ts:147) uses waitForNetworkPromises() (2 cycles) — barely enough, fails when Onyx operations schedule extra macrotasks.
Test 2 (MiddlewareTest.ts:341) uses only waitForBatchedUpdates() (1 cycle) — clearly insufficient for 2 sequential requests with middleware processing.
Note: SequentialQueue.waitForIdle() can't help here because unpause() calls flush(false) which skips resetting isReadyPromise, so it's already resolved from resetQueue().
What changes do you think we should make in order to solve the problem?
Add sufficient async cycle flushing in both tests to cover the full SequentialQueue processing chain for 2 sequential requests:
Test 1 (line 147): After SequentialQueue.unpause(), change await waitForNetworkPromises() → await waitForNetworkPromises() followed by an additional await waitForBatchedUpdates() to cover the extra async hops from middleware Onyx operations between the two fetches.
Test 2 (line 341): After SequentialQueue.unpause(), change await waitForBatchedUpdates() → await waitForNetworkPromises() followed by await waitForBatchedUpdates() (same as test 1, since it has the same two-request pattern).
This directly addresses the insufficient wait budget without introducing new synchronization primitives.
What alternative solutions did you explore?
Using SequentialQueue.waitForIdle(): Won't work because unpause() → flush(false) preserves the already-resolved isReadyPromise from resetQueue(), so waitForIdle() resolves immediately before processing starts.
Modifying flush() to always reset the promise: Would fix the test but changes production semantics — the shouldResetPromise=false path exists to prevent hanging READ requests during unpause (documented at line 271-274). Too risky for a test fix.
Next Steps for Contributor+ team: Reply with @MelvinBot implement this to create a draft PR, @MelvinBot <your feedback> to refine this analysis, or explain why you are rejecting Melvin's proposal.
@dmkt9 can you please accept the job below? Please reply here and tag me once you have.
If you're not sent an offer in Upwork when you're assigned to an issue, please tag me. If an Upwork job exists, please submit a proposal. https://www.upwork.com/jobs/~022066902892631038155
@dmkt9 can you please accept the job below? Please reply here and tag me once you have. If you're not sent an offer in Upwork when you're assigned to an issue, please tag me. If an Upwork job exists, please submit a proposal. upwork.com/jobs/~022066902892631038155
Activity
melvin-bot commentedon May 14, 2026
Triggered auto assignment to Contributor-plus team member for initial proposal review - @dmkt9 (
External)[-]Flaky test: MiddlewareTest › HandleUnusedOptimisticID - fetch call count assertion fails intermittently[/-][+][$250] Flaky test: MiddlewareTest › HandleUnusedOptimisticID - fetch call count assertion fails intermittently[/+]melvin-bot commentedon May 14, 2026
Job added to Upwork: https://www.upwork.com/jobs/~022054904870462893459
[-][$250] Flaky test: MiddlewareTest › HandleUnusedOptimisticID - fetch call count assertion fails intermittently[/-][+][$50] Flaky test: MiddlewareTest › HandleUnusedOptimisticID - fetch call count assertion fails intermittently[/+]melvin-bot commentedon May 14, 2026
Upwork job price has been updated to $50
mountiny commentedon May 14, 2026
Making this $50 as its flaky test
yusufdeveloper2903 commentedon May 14, 2026
🚨 Edited by proposal-police: This proposal was edited at 2026-05-14 12:46:58 UTC.
Proposal
What is the root cause of that problem?
Both flaky tests push two requests onto the
SequentialQueue, then assertexpect(global.fetch).toHaveBeenCalledTimes(2)after only a fixed number of microtask/macrotask flushes —waitForNetworkPromises()(twowaitForBatchedUpdatesrounds) in the first test, and a singlewaitForBatchedUpdates()in the second:App/tests/unit/MiddlewareTest.ts
Lines 146 to 149 in 1b89b63
App/tests/unit/MiddlewareTest.ts
Lines 340 to 343 in 1b89b63
But the queue does not process both requests in a fixed number of cycles.
process()handles one request at a time through a recursive chain — it resolves the fetch mock (two async hops for theasyncmock + itsasync json()), runs the middleware chain (which performs async Onyx writes), applies the response viaOnyx.update, and only then does.then(() => process())pick up the next request and fire its fetch:App/src/libs/Network/SequentialQueue.ts
Lines 165 to 196 in 1b89b63
The async depth required for the second
fetchto fire is variable. Under CI scheduling pressure it exceeds the fixed number of flushes, so the assertion runs while only the firstfetchhas been called →toHaveBeenCalledTimes(2)fails. The same unboundedprocess()chain still running after the test returns is also the open handle behind the "worker process failed to exit gracefully" teardown leak.Note: commit
a7ede572585improved this file's test isolation (beforeEachresets), but it did not address this assertion-timing race.What changes do you think we should make in order to solve the problem?
Stop waiting a fixed number of cycles and instead wait for the queue to actually drain, using the existing
SequentialQueue.waitForIdle()idiom (already used inReportAttachments,WorkspaceUpgradeTest,PaginationTest).push()of the first request synchronously callsflush(true), which resetsisReadyPromisebefore the test reaches its wait; that promise resolves inprocess().finally()only after the whole recursive chain has drained both requests:App/src/libs/Network/SequentialQueue.ts
Lines 583 to 589 in 1b89b63
Middleware › HandleUnusedOptimisticID › Request with preexistingReportID(L147):Middleware › HandleUnusedOptimisticID › OpenReport to a chat with preexistingReportID and clean up optimistic participant data(L341):(
SequentialQueueis already imported in this file; the trailingwaitForBatchedUpdates()in the second test keeps the follow-up Onyx reads consistent.) This makes thefetchcount deterministic and guarantees the queue is fully drained before teardown, which also removes the worker-exit leak.The identical fixed-cycle pattern exists at the other
SequentialQueue.unpause()sites in this file (L93-L94, L198-L199, L253-L254, L438-L439); applying the samewaitForIdle()wait there prevents the same flake from resurfacing in those tests.What alternative solutions did you explore? (Optional)
Adding more
waitForBatchedUpdates()cycles (or wideningwaitForNetworkPromises) — rejected: it only lowers the probability of the race, it does not eliminate it, since the queue's async depth is unbounded relative to any fixed cycle count.await SequentialQueue.getCurrentRequest()— rejected:currentRequestPromiseisnullsynchronously afterunpause()(the chain starts inside an asyncOnyx.connectWithoutViewcallback), so awaiting it immediately resolves and waits for nothing.MelvinBot commentedon May 14, 2026
Proposal
What is the root cause of that problem?
The two failing tests wait a fixed number of async cycles before asserting on
global.fetchcall count, butSequentialQueuerequires a variable number of async hops between processing request #1 and issuing fetch #2.Specifically:
waitForNetworkPromises()flushes exactly 2 microtask+timer cycleswaitForBatchedUpdates()flushes only 1 cycleBetween the two fetches, the queue chains through:
fetch()→response.json()→ middleware processing (HandleUnusedOptimisticID + SaveResponseInOnyx with Onyx writes) →endPersistedRequestAndRemoveFromQueue(another Onyx write) → recursiveprocess()→ fetch #2. That's 5-6+ async hops.Test 1 (
MiddlewareTest.ts:147) useswaitForNetworkPromises()(2 cycles) — barely enough, fails when Onyx operations schedule extra macrotasks.Test 2 (
MiddlewareTest.ts:341) uses onlywaitForBatchedUpdates()(1 cycle) — clearly insufficient for 2 sequential requests with middleware processing.Note:
SequentialQueue.waitForIdle()can't help here becauseunpause()callsflush(false)which skips resettingisReadyPromise, so it's already resolved fromresetQueue().What changes do you think we should make in order to solve the problem?
Add sufficient async cycle flushing in both tests to cover the full SequentialQueue processing chain for 2 sequential requests:
Test 1 (line 147): After
SequentialQueue.unpause(), changeawait waitForNetworkPromises()→await waitForNetworkPromises()followed by an additionalawait waitForBatchedUpdates()to cover the extra async hops from middleware Onyx operations between the two fetches.Test 2 (line 341): After
SequentialQueue.unpause(), changeawait waitForBatchedUpdates()→await waitForNetworkPromises()followed byawait waitForBatchedUpdates()(same as test 1, since it has the same two-request pattern).This directly addresses the insufficient wait budget without introducing new synchronization primitives.
What alternative solutions did you explore?
SequentialQueue.waitForIdle(): Won't work becauseunpause()→flush(false)preserves the already-resolvedisReadyPromisefromresetQueue(), sowaitForIdle()resolves immediately before processing starts.flush()to always reset the promise: Would fix the test but changes production semantics — theshouldResetPromise=falsepath exists to prevent hanging READ requests during unpause (documented at line 271-274). Too risky for a test fix.Next Steps for Contributor+ team: Reply with
@MelvinBot implement thisto create a draft PR,@MelvinBot <your feedback>to refine this analysis, or explain why you are rejecting Melvin's proposal.71 remaining items
melvin-bot commentedon Jun 11, 2026
Payment Summary
Upwork Job
BugZero Checklist (@mallenexpensify)
melvin-bot commentedon Jun 11, 2026
@mallenexpensify Whoops! This issue is 2 days overdue. Let's get this updated quick!
mallenexpensify commentedon Jun 15, 2026
Payment Summary
Contributor+: @dmkt9 paid $125 via Upwork
$125 cuz tests weren't needed.
dmkt9 commentedon Jun 16, 2026
Hi @mallenexpensify, I'm still receiving payment through Upwork.
mallenexpensify commentedon Jun 16, 2026
@dmkt9 can you please accept the job below? Please reply here and tag me once you have.
If you're not sent an offer in Upwork when you're assigned to an issue, please tag me. If an Upwork job exists, please submit a proposal.
https://www.upwork.com/jobs/~022066902892631038155
dmkt9 commentedon Jun 16, 2026
@mallenexpensify Thanks. I've accepted the offer
mallenexpensify commentedon Jun 16, 2026
@dmkt9 paid, summary updated above. Thx.