Skip to content

test(ci): classify e2e_token_bridge_tutorial L1-tx confirmation timeout as the known flake - #24385

Merged
spalladino merged 1 commit into
merge-train/spartan-v5from
cb/fix-merge-train-spartan-v5-anvil
Jun 29, 2026
Merged

test(ci): classify e2e_token_bridge_tutorial L1-tx confirmation timeout as the known flake#24385
spalladino merged 1 commit into
merge-train/spartan-v5from
cb/fix-merge-train-spartan-v5-anvil

Conversation

@AztecBot

Copy link
Copy Markdown
Collaborator

What

Extends the existing .test_patterns.yml flake entry for
src/composed/e2e_token_bridge_tutorial_test.test.ts so it also matches the
viem WaitForTransactionReceiptTimeoutError confirmation-timeout failure mode,
not just the jest per-test timeout.

Why — the merge-train/spartan-v5 CI red on this branch

The train CI run failed on this one test:

FAIL src/composed/e2e_token_bridge_tutorial_test.test.ts
  ✕ Deploys tokens & bridges to L1 & L2, mints & publicly bridges tokens (189531 ms)
    WaitForTransactionReceiptTimeoutError: Timed out while waiting for transaction
    with hash "0x01f3fb09…" to be confirmed.
    Version: viem@2.38.2

(test log: http://ci.aztec-labs.com/1643e817d7242e0e)

This test is already known-flaky — there is an entry for it owned by @spalladino — but its error_regex only covered Exceeded timeout of N ms for a test (the jest per-test budget). In this run the test failed faster (~189s of its 900s budget) because viem's own per-tx confirmation timeout fired first, producing a different error string. The flake matcher (grep -E over the test log in ci3/get_test_entry) therefore did not classify it, so the failure went red and blocked the train.

The L1↔L2 bridging flow sends a series of L1 txs against an automining anvil that the local-network sequencer and cheat codes also drive concurrently. Under load a tx can sit unmined past viem's confirmation window — the same automine-race class, surfacing here in the test's own L1 txs. Both observed symptoms (jest timeout, viem confirmation timeout) are the same flake.

Not caused by #24378

The train HEAD is #24378 (broadcast L1 deploy txs one at a time on anvil), but that change only touches l1-contracts/scripts/forge_broadcast.js (the L1 contract deploy path). In the failing run that deploy succeeded ([forge_broadcast] Broadcast succeeded at 20:52:19); the failure is ~3.5 min later in the test body, which never goes through forge_broadcast.js. In every version (pre-#24316, #24316, #24378) anvil ends in automine after deploy, so #24378 does not change the test-body mining environment. #24378 was simply the train HEAD when this pre-existing flake surfaced in a mode the pattern didn't catch.

Verification

A full ./bootstrap.sh ci run is neither meaningful nor feasible for a flake-classification config change (and the test is non-deterministic by nature). The meaningful check is that ci3/get_test_entry now classifies the observed failure:

  • yq parses the updated entry cleanly.
  • get_test_entry "<compose token_bridge_tutorial cmd>" <log-with-WaitForTransactionReceiptTimeoutError> now returns the entry (→ flake, non-blocking).
  • A negative control (an unrelated error line) still returns nothing, so the broadened regex stays scoped to this test's timeout flake.

Created by claudebox · group: slackbot

@AztecBot AztecBot added ci-draft Run CI on draft PRs. ci-no-fail-fast Sets NO_FAIL_FAST in the CI so the run is not aborted on the first failure claudebox Owned by claudebox. it can push to this PR. labels Jun 29, 2026
@spalladino
spalladino marked this pull request as ready for review June 29, 2026 21:11
@spalladino
spalladino enabled auto-merge (squash) June 29, 2026 21:11
@spalladino
spalladino merged commit 9756edd into merge-train/spartan-v5 Jun 29, 2026
63 of 71 checks passed
@spalladino
spalladino deleted the cb/fix-merge-train-spartan-v5-anvil branch June 29, 2026 21:16
spalladino pushed a commit that referenced this pull request Jul 2, 2026
…encer nonce race (#24386)

## Root cause

`e2e_token_bridge_tutorial_test` flakes with
`WaitForTransactionReceiptTimeoutError: Timed out while waiting for
transaction … to be confirmed`
([example](http://ci.aztec-labs.com/1643e817d7242e0e)). It is **not**
caused by #24378 — that PR only touches `forge_broadcast.js` (the L1
*contract deploy*), which succeeds in the failing run; the timeout is
~3.5 min later on one of the test's own L1 txs.

The test runs against `aztec start --local-network`, whose
`AutomineSequencer` continuously publishes checkpoint txs to L1. In
`local-network.ts` the sequencer-publisher and validator keys are
derived from `DefaultMnemonic` (`'test test … junk'`) at **address index
0** — `0xf39Fd6…` (visible as the publisher in the log).

The test's L1 client was created with the **same** mnemonic and the
**same** default index 0:

```ts
const l1Client = createExtendedL1Client(ETHEREUM_HOSTS.split(','), MNEMONIC); // → account index 0
```

So the test and the node's sequencer send L1 txs from one account,
sharing a single nonce sequence. Against an automining anvil they race:
a test tx can land with a nonce the sequencer just consumed (dropped /
"already known"), or with a future-nonce gap that automine won't mine
until the gap fills. Either way the tx gets a hash but never confirms,
and viem's per-tx confirmation timeout fires. This is intermittent
because it only bites when a test tx and a sequencer publish overlap.

## Fix

Derive the test's L1 client from a **different** address index so its
nonce space is independent of the sequencer's:

```ts
const l1Client = createExtendedL1Client(ETHEREUM_HOSTS.split(','), mnemonicToAccount(MNEMONIC, { addressIndex: 1 }));
```

Index 1 (`0x70997970…`) is funded by anvil's default mnemonic and is
unused by the local-network node (publisher/validator = index 0; index 2
is the prover and 3+ are attesters by the existing `setup.ts` /
`setup_p2p_test.ts` convention, none of which run in local-network). The
test is fully self-contained on its own L1 account — it deploys and owns
the `TestERC20`, `FeeAssetHandler` and `TokenPortal`, and bridges
to/from `l1Client.account.address` — so nothing requires it to be
account 0. All of the test's L1 txs (including the `L1TokenManager` /
`L1TokenPortalManager`, built from `l1Client`) now go through index 1.

## Verification

This is an `compose` e2e test that needs docker + a full network; the
container here is a source-only checkout (no `node_modules`, no docker),
so I could not run the test or `./bootstrap.sh ci` locally — flagging
that explicitly per the repo's red/green policy. The fix is verified by
analysis: the timed-out tx is one of the test's own L1 txs, the
shared-account nonce race is the established cause of this exact viem
symptom, and account index 0 vs 1 are the well-known distinct anvil
accounts (idx 0 = `0xf39Fd6…`, the publisher in the log; idx 1 =
`0x70997970…`).

## Relationship to #24385

#24385 broadened this test's `.test_patterns.yml` flake pattern to keep
CI from going red on the residual flake. This PR removes the underlying
cause. The flake entry can stay as a safety net, or be narrowed back to
the jest-timeout-only form once this has soaked — happy to follow up
either way.


---
*Created by
[claudebox](https://claudebox.work/v2/sessions/c01cbe3dd422bf00) ·
group: `slackbot`*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-draft Run CI on draft PRs. ci-no-fail-fast Sets NO_FAIL_FAST in the CI so the run is not aborted on the first failure claudebox Owned by claudebox. it can push to this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants