test(e2e): allocate HA node p2p ports above the ephemeral range to deflake e2e_ha_full - #24418
Merged
Merged
Conversation
…flake e2e_ha_full
The HA full suite assigned fixed p2p ports `40400 + i + 1` (40400-40405) to its
bootstrap and HA nodes. Those ports sit inside the Linux default ephemeral range
(32768-60999), which the OS draws from for the in-process prover node (it listens
on p2pPort 0) and for outbound connections. When an ephemeral socket already held
one of those ports at bind time, the corresponding node's libp2p TCP listener
failed to bind and threw `ERR_NO_VALID_ADDRESSES` (libp2p's wrapper around the
swallowed EADDRINUSE), aborting `beforeAll`. jest then attributed the failure to
the first test in the suite ("should not be affected by process.env.TZ changes")
and skipped the rest, making the flake masquerade as a timezone bug.
Allocate every node's port via get-port from 61000-65535, above the ephemeral
range, so neither in-process ephemeral sockets nor concurrent CI jobs can hold a
node's port. Set p2pBroadcastPort alongside p2pPort because discv5 mutates the
config in place to default the broadcast port, and that value would otherwise leak
from the bootstrap config into the HA nodes' configs (built by spreading it).
PhilWindle
approved these changes
Jul 1, 2026
rangozd
pushed a commit
to rangozd/aztec-packages
that referenced
this pull request
Aug 5, 2026
BEGIN_COMMIT_OVERRIDE refactor(e2e): relocate no-node straggler tests (AztecProtocol#24344) test(e2e): pin gas_estimation public-payment txs to one block to deflake fee comparison (AztecProtocol#24382) feat(archiver): event-trigger L2BlockStream sync from archiver updates (AztecProtocol#24317) test(e2e): speed up individual e2e tests (AztecProtocol#24345) chore(e2e): warm blob KZGs in parallel during setup (AztecProtocol#24383) chore: add perf as a valid PR title prefix (AztecProtocol#24412) test(e2e): adopt shared wait helpers (AztecProtocol#24404) test(e2e): run prover client.test.ts in CI (AztecProtocol#24399) fix(ethereum): mine empty L1 blocks without touching the mempool (AztecProtocol#24414) test(e2e): deflake empty block proving test (AztecProtocol#24411) test(e2e): allocate HA node p2p ports above the ephemeral range to deflake e2e_ha_full (AztecProtocol#24418) fix(sequencer): use evmMine in automine auto-settle to avoid dropping test L1 txs (AztecProtocol#24421) test(e2e): instrument common spans for wall-clock tracking (AztecProtocol#24407) test(e2e): remove redundant reqresp_no_handshake e2e test (AztecProtocol#24424) END_COMMIT_OVERRIDE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Deflakes
e2e_ha_full.parallel.test.ts, which FLAKED in CI (run 6d29cd3fcf374a68) reportingshould not be affected by process.env.TZ changesas the failing test.Root cause
beforeAllwhile creating the second HA node (HA-1):CodeError: Transport (@libp2p/tcp) could not listen on any available address, codeERR_NO_VALID_ADDRESSES. jest attributed thebeforeAllthrow to the first test in the suite (the timezone test, a pure Postgres test that never ran) and skipped the other 7.(config.p2pPort ?? 40400) + i + 1, i.e. 40400 (bootstrap), 40401-40405 (HA nodes). These sit inside the Linux default ephemeral port range (32768-60999, per/proc/sys/net/ipv4/ip_local_port_range).setup()withp2pPort: 0) and for outbound TCP connections. When an ephemeral socket already held one of 40401-40405 at bind time, that HA node's libp2p TCPlisten()failed withEADDRINUSE. libp2p runs the listen attempt underPromise.allSettledand, finding no fulfilled result, throws the aggregateERR_NO_VALID_ADDRESSESwithout surfacing the underlyingEADDRINUSE— which is why noEADDRINUSEstring appears in the logs.061f6753f56b00fd): identical fixed ports, but there HA-0..HA-4 bound 40401-40405 cleanly. The only divergence is HA-1 failing to bind 40402 in the failed run.Fix
get-portfrom 61000-65535, above the ephemeral range, so neither in-process ephemeral sockets nor concurrent CI jobs can be holding a node's port when libp2p binds it. Applied to the bootstrap node (viasetup()opts) and to each HA node.p2pBroadcastPortalongsidep2pPort: discv5 defaults the broadcast port by mutating the config object in place, and that mutated value would otherwise leak from the bootstrap config into the HA node configs (which are built by spreadingconfig), making them advertise the wrong port in their ENR.Verification
@aztec/end-to-endpasses; prettier clean.ERR_NO_VALID_ADDRESSES) and source (fixed ports inside the ephemeral range;p2pPort: 0prover node; discv5 binding UDP on the same port) — moving the ports above the ephemeral range removes the collision source entirely.