test: deflake e2e_offchain_payment reorg reprocessing race - #24309
Merged
spalladino merged 1 commit intoJun 25, 2026
Conversation
The 'reprocesses an offchain-delivered payment after an L1 reorg' test asserts Alice's balance is the rolled-back mint (100) after forceReorg, but reads 60 intermittently. revertToCheckpoint prunes the block and restores the un-mined transfer to the pending pool; the AutomineSequencer's ~50ms mempool poller then re-mines it autonomously, and because the PXE auto-syncs on every simulate, the post-reorg balance read races whether the PXE has caught up to the re-mined block. Pause the AutomineSequencer before revertToCheckpoint so its poller cannot win the re-mine race, stay paused through the explicit forceEmptyBlock re-mine (which still drains the pending pool), and resume in afterEach. Closes the race without relaxing the behavioral aliceAfterRollback == 100 assertion.
mverzilli
approved these changes
Jun 25, 2026
spalladino
enabled auto-merge (squash)
June 25, 2026 18:24
Collaborator
Flakey Tests🤖 says: This CI run detected 2 tests that failed, but were tolerated due to a .test_patterns.yml entry. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
e2e_offchain_payment.test.ts› "reprocesses an offchain-delivered payment after an L1 reorg" flakes (CI run7d8ad07c73ce84e2) with:60n == 100 - 40: the reverted transfer's effect was still visible to the PXE when the post-reorg balance was read, instead of the rolled-back full mint of100n.Root cause
A race between the
AutomineSequencer's automatic re-mine of the restored transfer tx and the test's post-reorg assertions:forceReorg→revertToCheckpoint(6)prunes block 7 and restores the un-mined transfer to the pending pool (p2pClient.sync()inrunRevert,automine_sequencer.ts:629), then returns. It does not itself re-mine.automine_sequencer.ts:175, defaultpollIntervalMs50ms) whosebuildIfPendingautonomously re-mines block 7 from the restored pool.autoSync: true, soget_balance(...).simulate()re-syncs before every read. Whether Alice's read at line 191 returns100n(PXE at pruned tip, block 6) or60n(PXE re-synced to the re-mined block 7) depends purely on this timing.The failed log shows
Updated pxe last block to 7landing between Bob's read (passes) and Alice's read (fails); the passed log keeps the PXE pinned at block 6 across both reads. Both runs showBuilding automine checkpoint {checkpointNumber:7, txCount:1}, confirming the poller drives the re-mine.Fix (test only)
yarn-project/end-to-end/src/e2e_offchain_payment.test.ts:forceReorg:automine.pause()beforerevertToCheckpoint.pause()gates only the mempool poller; explicit ops (revertToCheckpoint,buildEmptyBlock) still run. Pausing before the transfer is restored to the pool means the poller can never enqueue an auto re-mine, so the post-reorg assertions observe a stable pruned state.forceEmptyBlock().buildEmptyBlock(allowEmpty:true) permits an empty block but still drains the pending pool, so it deterministically re-mines the restored transfer with no poller racing it. The existingretryUntilstill waits for the PXE re-sync, so the reprocessing behavior under test is preserved.afterEachthat callsgetAutomineSequencer()?.resume()(idempotent), so a pause is never left dangling.This removes the nondeterministic auto re-mine that raced the assertions rather than relaxing the
aliceAfterRollback == 100assertion — which is behavioral (it verifies the reorg rolled Alice's note state back) and is kept. Not a skip, not a.test_patterns.ymlentry.Verification
yarn build: exit 0, no type errors.paused;buildEmptyBlockstill drains pending), and was cross-checked to confirm it does not break the laterretryUntil.