test(ci): classify e2e_token_bridge_tutorial L1-tx confirmation timeout as the known flake - #24385
Merged
spalladino merged 1 commit intoJun 29, 2026
Conversation
…ut as the known flake
spalladino
approved these changes
Jun 29, 2026
spalladino
marked this pull request as ready for review
June 29, 2026 21:11
spalladino
enabled auto-merge (squash)
June 29, 2026 21:11
This was referenced Jun 29, 2026
spalladino
pushed a commit
that referenced
this pull request
Jul 2, 2026
…encer nonce race (#24386) ## Root cause `e2e_token_bridge_tutorial_test` flakes with `WaitForTransactionReceiptTimeoutError: Timed out while waiting for transaction … to be confirmed` ([example](http://ci.aztec-labs.com/1643e817d7242e0e)). It is **not** caused by #24378 — that PR only touches `forge_broadcast.js` (the L1 *contract deploy*), which succeeds in the failing run; the timeout is ~3.5 min later on one of the test's own L1 txs. The test runs against `aztec start --local-network`, whose `AutomineSequencer` continuously publishes checkpoint txs to L1. In `local-network.ts` the sequencer-publisher and validator keys are derived from `DefaultMnemonic` (`'test test … junk'`) at **address index 0** — `0xf39Fd6…` (visible as the publisher in the log). The test's L1 client was created with the **same** mnemonic and the **same** default index 0: ```ts const l1Client = createExtendedL1Client(ETHEREUM_HOSTS.split(','), MNEMONIC); // → account index 0 ``` So the test and the node's sequencer send L1 txs from one account, sharing a single nonce sequence. Against an automining anvil they race: a test tx can land with a nonce the sequencer just consumed (dropped / "already known"), or with a future-nonce gap that automine won't mine until the gap fills. Either way the tx gets a hash but never confirms, and viem's per-tx confirmation timeout fires. This is intermittent because it only bites when a test tx and a sequencer publish overlap. ## Fix Derive the test's L1 client from a **different** address index so its nonce space is independent of the sequencer's: ```ts const l1Client = createExtendedL1Client(ETHEREUM_HOSTS.split(','), mnemonicToAccount(MNEMONIC, { addressIndex: 1 })); ``` Index 1 (`0x70997970…`) is funded by anvil's default mnemonic and is unused by the local-network node (publisher/validator = index 0; index 2 is the prover and 3+ are attesters by the existing `setup.ts` / `setup_p2p_test.ts` convention, none of which run in local-network). The test is fully self-contained on its own L1 account — it deploys and owns the `TestERC20`, `FeeAssetHandler` and `TokenPortal`, and bridges to/from `l1Client.account.address` — so nothing requires it to be account 0. All of the test's L1 txs (including the `L1TokenManager` / `L1TokenPortalManager`, built from `l1Client`) now go through index 1. ## Verification This is an `compose` e2e test that needs docker + a full network; the container here is a source-only checkout (no `node_modules`, no docker), so I could not run the test or `./bootstrap.sh ci` locally — flagging that explicitly per the repo's red/green policy. The fix is verified by analysis: the timed-out tx is one of the test's own L1 txs, the shared-account nonce race is the established cause of this exact viem symptom, and account index 0 vs 1 are the well-known distinct anvil accounts (idx 0 = `0xf39Fd6…`, the publisher in the log; idx 1 = `0x70997970…`). ## Relationship to #24385 #24385 broadened this test's `.test_patterns.yml` flake pattern to keep CI from going red on the residual flake. This PR removes the underlying cause. The flake entry can stay as a safety net, or be narrowed back to the jest-timeout-only form once this has soaked — happy to follow up either way. --- *Created by [claudebox](https://claudebox.work/v2/sessions/c01cbe3dd422bf00) · group: `slackbot`*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Extends the existing
.test_patterns.ymlflake entry forsrc/composed/e2e_token_bridge_tutorial_test.test.tsso it also matches theviem
WaitForTransactionReceiptTimeoutErrorconfirmation-timeout failure mode,not just the jest per-test timeout.
Why — the
merge-train/spartan-v5CI red on this branchThe train CI run failed on this one test:
(test log: http://ci.aztec-labs.com/1643e817d7242e0e)
This test is already known-flaky — there is an entry for it owned by @spalladino — but its
error_regexonly coveredExceeded timeout of N ms for a test(the jest per-test budget). In this run the test failed faster (~189s of its 900s budget) because viem's own per-tx confirmation timeout fired first, producing a different error string. The flake matcher (grep -Eover the test log inci3/get_test_entry) therefore did not classify it, so the failure went red and blocked the train.The L1↔L2 bridging flow sends a series of L1 txs against an automining anvil that the local-network sequencer and cheat codes also drive concurrently. Under load a tx can sit unmined past viem's confirmation window — the same automine-race class, surfacing here in the test's own L1 txs. Both observed symptoms (jest timeout, viem confirmation timeout) are the same flake.
Not caused by #24378
The train HEAD is #24378 (
broadcast L1 deploy txs one at a time on anvil), but that change only touchesl1-contracts/scripts/forge_broadcast.js(the L1 contract deploy path). In the failing run that deploy succeeded ([forge_broadcast] Broadcast succeededat 20:52:19); the failure is ~3.5 min later in the test body, which never goes throughforge_broadcast.js. In every version (pre-#24316, #24316, #24378) anvil ends in automine after deploy, so #24378 does not change the test-body mining environment. #24378 was simply the train HEAD when this pre-existing flake surfaced in a mode the pattern didn't catch.Verification
A full
./bootstrap.sh cirun is neither meaningful nor feasible for a flake-classification config change (and the test is non-deterministic by nature). The meaningful check is thatci3/get_test_entrynow classifies the observed failure:yqparses the updated entry cleanly.get_test_entry "<compose token_bridge_tutorial cmd>" <log-with-WaitForTransactionReceiptTimeoutError>now returns the entry (→ flake, non-blocking).Created by claudebox · group:
slackbot