Skip to content

test(e2e): consolidate the automine test category - #24343

Merged
spalladino merged 14 commits into
merge-train/spartan-v5from
spl/e2e-automine
Jun 29, 2026
Merged

spalladino merged 14 commits into
merge-train/spartan-v5from
spl/e2e-automine

Conversation

@spalladino

@spalladino spalladino commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Motivation

The automine category is the largest remaining e2e group: a single in-process node running the
deterministic AutomineSequencer (one block per submitted tx, synchronous L1 publish, no committee,
prover, or validator), opted into via the AUTOMINE_E2E_OPTS preset. It was spread across 4 standalone
domain base classes (28 tests) and ~42 root e2e_*.test.ts files, each repeating the same
{ ...AUTOMINE_E2E_OPTS } spread at the call site. This consolidates them under one category base and a
behavior-organized tree, mirroring the single-node/multi-node work this PR is stacked on.

Stacked on spl/e2e-consolidation-2 (PR for the single-node/multi-node/infra consolidation).

Approach

A new automine/automine_test_context.ts is the category base, a sibling of SingleNodeTestContext:
both wrap fixtures/setup.ts:setup(), but AutomineTestContext fixes the automine topology and makes
AUTOMINE_E2E_OPTS (plus fundSponsoredFPC) the default, so tests pass numberOfAccounts instead of
spreading the preset. It exposes the common handles, a static setup<T> factory, markProvenAndWarp
(the documented mark-proven-then-warp idiom), registerContract, and applyManualParentChild.

The two thin domain bases (DeployTest, NestedContractTest) fold into the base; the two token bases
(TokenContractTest, BlacklistTokenContractTest) become composed harnesses that extend AutomineTestContext and run their TokenSimulator/snapshot/role-delay setup after super.setup(). Root
tests are routed through the base and organized by behavior. Test bodies and assertions are preserved
verbatim through every conversion.

The commits map to the plan's PR-split:

  • add AutomineTestContext base + helpers — base context, README, blanket flake group.
  • fold deploy + nested contract tests into automine — deploy/nested tests under
    automine/contracts/{deploy,nested}/, e2e_storage_proof under automine/contracts/.
  • convert token + blacklist tests to automine harnesses — both token bases become harnesses; 17 test
    files + 2 harnesses under automine/token/.
  • move automine root tests + recategorize snapshot_sync — the 42 root tests bucketed under
    automine/{smoke,token,contracts,accounts,notes,execution,lifecycle}/.
  • re-bucket automine second level — review found execution/ and lifecycle/ had become grab-bags, so
    they are dissolved: notes/ is renamed to effects/ (and gains pruned_blocks), a new simulation/
    folder holds the AVM simulator / kernelless simulation / circuit recorder, the ABI-surface tests move
    into contracts/, and the genuine miscellany is flattened to the automine/ root. A second-level
    folder is created only when it earns its keep.
  • document the test category structure in a top-level README — rewrites end-to-end/README.md with a
    human overview of the five topology categories plus a detailed agent reference (base-class hierarchy,
    the hydrateFromContext harness pattern, the .parallel rule, CI discovery, and the relocated no-node
    tests). The relocated tests it points to land in the sibling no-node-relocations PR.

Hierarchy before → after

Before (root + domain dirs):

src/e2e_token_contract/         (token_contract_test.ts + 10 tests)
src/e2e_blacklist_token_contract/ (blacklist_token_contract_test.ts + 7 tests)
src/e2e_nested_contract/        (nested_contract_test.ts + 4 tests)
src/e2e_deploy_contract/        (deploy_test.ts + 4 tests)
src/e2e_storage_proof/
src/e2e_*.test.ts               (42 root automine tests + e2e_snapshot_sync)

After:

src/automine/
  automine_test_context.ts   README.md
  token/        token + blacklist harnesses, the 17 token/blacklist tests,
                amm lending nft orderbook crowdfunding_and_claim escrow
  contracts/    deploy/ nested/ + contract_updates static_calls nested_utility_calls
                storage_proof abi_types option_params state_vars (+ fixtures/)
  accounts/     account_contracts keys multiple_accounts_1_enc_key 2_pxes authwit scope_isolation
  effects/      note_getter pending_note_hashes partial_notes event_logs event_only
                offchain_effect offchain_payment large_public_event custom_message
                tx_effect_oracle pruned_blocks
  simulation/   avm_simulator kernelless_simulation circuit_recorder
  <root flat>:  smoke ordering double_spend phase_check mempool_limit card_game
                private_voting expiration_timestamp genesis_timestamp pxe

A second-level folder is created only when it earns its keep (a shared harness, an existing
sub-hierarchy, or a coherent domain of several files); the remaining miscellaneous protocol/execution
behaviors live as flat files directly under automine/.

Files with more than one top-level it carry the .parallel suffix so CI splits each it into its own
job. e2e_cheat_codes (no setup()) stays at root as a Phase 6 unit/integration relocation candidate.

Helpers

  • markProvenAndWarp(seconds) on the base composes cheatCodes.rollup.markAsProven() then
    warpL2TimeAtLeastBy(...) with the ordering required under automine (a long warp crosses many epochs
    with no proofs; without marking proven first, the pruning window resets the tip to genesis and the
    warp's own empty-checkpoint propose fails with Rollup__InvalidArchive). Replaces the verbatim copies
    in BlacklistTokenContractTest.crossTimestampOfChange and the prune/update root tests.
  • registerContract and the StatefulContractCtorArgs/ContractArtifactClass aliases lift from
    DeployTest onto the base; applyManualParentChild lifts from NestedContractTest.applyManual.

Local verification: full yarn build, yarn lint end-to-end, and yarn format --check are green after
each commit, and the test_cmds / compat_test_cmds generators list every automine file exactly once
(avm_simulator runs in test_cmds via the bespoke AVM-dump line and in compat as a regular test;
kernelless_simulation is excluded from compat only). The
e2e tests themselves could not be run in this environment — the setup hook hits the 5-minute timeout on
untouched root automine tests too — so behavior verification is deferred to CI.

Part of A-1176

@spalladino spalladino added the S-do-not-merge Status: Do not merge this PR label Jun 26, 2026
@spalladino spalladino changed the title refactor(e2e): consolidate the automine test category test(e2e): consolidate the automine test category Jun 26, 2026
@spalladino
spalladino force-pushed the spl/e2e-consolidation-2 branch from f5a11ea to 94e41a4 Compare June 26, 2026 20:55
@spalladino
spalladino force-pushed the spl/e2e-automine branch 2 times, most recently from 66ed168 to dbbbf39 Compare June 26, 2026 22:32
Base automatically changed from spl/e2e-consolidation-2 to merge-train/spartan-v5 June 29, 2026 07:20
@AztecBot

AztecBot commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Flakey Tests

🤖 says: This CI run detected 2 tests that failed, but were tolerated due to a .test_patterns.yml entry.

\033FLAKED\033 (8;;http://ci.aztec-labs.com/95473d2943d545f7�95473d2943d545f78;;�):  yarn-project/end-to-end/scripts/run_test.sh simple src/single-node/fees/gas_estimation.parallel.test.ts "estimates gas with public payment method" (288s) (code: 0) group:e2e-p2p-epoch-flakes
\033FLAKED\033 (8;;http://ci.aztec-labs.com/8ee0a3c7392ed970�8ee0a3c7392ed9708;;�):  yarn-project/end-to-end/scripts/run_test.sh simple src/automine/contracts/deploy/private_initialization.parallel.test.ts "refuses to call a public function enqueued before private initialization in same tx" (17s) (code: 0) group:e2e-p2p-epoch-flakes

@spalladino spalladino removed the S-do-not-merge Status: Do not merge this PR label Jun 29, 2026
@spalladino
spalladino enabled auto-merge (squash) June 29, 2026 17:20
Add the automine category base context as a sibling of SingleNodeTestContext:
it wraps fixtures/setup.ts:setup() with AUTOMINE_E2E_OPTS as the default, exposes
the common test handles, and provides markProvenAndWarp, registerContract, and
applyManualParentChild. Lands unused; the test moves follow in later commits.

Adds the blanket automine flake group to .test_patterns.yml.
Fold DeployTest and NestedContractTest into AutomineTestContext: registerContract
and the StatefulContractCtorArgs/ContractArtifactClass aliases move onto the base,
and applyManual becomes applyManualParentChild. Move the deploy tests under
automine/contracts/deploy/, the nested tests under automine/contracts/nested/, and
e2e_storage_proof under automine/contracts/ (with its fixtures). Files with more
than one top-level it gain the .parallel suffix.

Adds the automine/contracts globs to both bootstrap.sh test-list arrays, extends
the compat flatten branch to cover src/automine/*, repoints the storage_proof_fetcher
eslint override, the snapshot-regen script, and the protocol-contracts fixture
provenance comment.
Make TokenContractTest and BlacklistTokenContractTest extend AutomineTestContext:
they keep their TokenSimulator, opt-in base/mint snapshots, and the Role helper, and
run their domain setup after super.setup(). crossTimestampOfChange now delegates to
the base markProvenAndWarp, and the harness loggers move to the e2e:automine:token /
e2e:automine:blacklist namespaces. Move the 17 token/blacklist test files plus the two
harnesses under automine/token/ (blacklist files gain a blacklist_ prefix to avoid
collisions), switching each from setup({ ...AUTOMINE_E2E_OPTS }) to the base default
and naming each describe after its path. Files with more than one top-level it gain
the .parallel suffix.

Adds the automine/token glob to both bootstrap.sh arrays, repoints the access_control
flake entry, and updates the e2e_persistence harness import.
Move the 42 root AUTOMINE_E2E_OPTS tests under automine/ bucketed by behavior
(smoke, token, contracts, accounts, notes, execution, lifecycle), switching each
from setup(N, { ...AUTOMINE_E2E_OPTS }) to AutomineTestContext.setup({
numberOfAccounts: N }) and naming each describe after its path. Files with more
than one top-level it gain the .parallel suffix.

Preserve the e2e_avm_simulator AVM-dump bespoke CI line (re-pointed at
src/automine/execution/avm_simulator.test.ts, dump name kept) with the file excluded
from the generic execution glob, and keep kernelless_simulation excluded from the
compat glob against its new path. Add the new automine subfolder globs to both
bootstrap.sh arrays and repoint the lending / offchain_payment flake entries.

Recategorize e2e_snapshot_sync: its checkpoint-history and corrupted-snapshot-fallback
assertions rely on interval block production that the AutomineSequencer does not provide
without injected txs, so per the plan's fallback it stays on the production sequencer and
moves to single-node/sync/ alongside synching rather than forcing a flaky automine
conversion.
The execution/ and lifecycle/ buckets had become grab-bags, so they are
dissolved. notes/ is renamed to effects/ (note discovery, events, offchain
effects) and gains pruned_blocks (note rediscovery after a prune). A new
simulation/ folder holds avm_simulator, kernelless_simulation, and the
circuit recorder. The ABI/storage-surface tests (abi_types, option_params,
state_vars) move into contracts/. The genuine miscellany (smoke, ordering,
double_spend, phase_check, mempool_limit, card_game, private_voting,
expiration_timestamp, genesis_timestamp, pxe) is flattened to the automine/
root: a second-level folder is created only when it earns its keep.

CI test discovery in bootstrap.sh and the .test_patterns.yml path entry are
updated to match; per-file describe titles are repointed to their new paths.
The compat_test_cmds simulation glob excluded both avm_simulator and
kernelless_simulation, but the bespoke AVM-dump line that covers
avm_simulator only exists in test_cmds. The previous execution/ compat
glob excluded only kernelless, so avm_simulator ran in compat as a
regular test; restore that by excluding only kernelless.
Rewrites end-to-end/README.md to serve two audiences: a succinct human
overview (the five topology categories, a where-does-my-test-go guide,
quick-start commands) and a detailed agent reference tail (base-class
hierarchy, the hydrateFromContext harness pattern, the .parallel rule,
CI discovery via bootstrap.sh's two arrays incl. the avm/kernelless
special-cases, .test_patterns.yml, compose/HA run modes, relocated
no-node tests, and support dirs). Preserves the legacy-artifacts section.

The relocated no-node tests it references land in the sibling
no-node-relocations PR.
The automine consolidation moved e2e_card_game.test.ts to
automine/card_game.test.ts and e2e_crowdfunding_and_claim.test.ts to
automine/token/crowdfunding_and_claim.parallel.test.ts. Update the
references array and the two #include_code directives in call_types.md
that still pointed at the old paths, which broke the docs build.
The 'failure cases' block asserts that set_admin / set_minter from
t.adminAddress revert once it is no longer admin, which only held when
the earlier 'Set admin' test had run. CI runs each test in isolation
(--testNamePattern), so 'Set admin' was skipped, t.adminAddress was
still admin, and the call succeeded instead of reverting. Move admin to
account1 in the block's own beforeAll (idempotent) so the failure cases
pass whether run alone or as part of the full suite.
The automine consolidation also moved e2e_deploy_contract/deploy_method.test.ts
to automine/contracts/deploy/deploy_method.parallel.test.ts and
e2e_event_logs.test.ts to automine/effects/event_logs.test.ts. Update the
eight #include_code directives in how_to_deploy_contract.md and
how_to_read_data.md that still pointed at the old paths; the docs build
aborts on the first missing include, so these surfaced only after the
call_types.md fix landed.
The .parallel split runs each top-level it() in its own container via
--testNamePattern, but 'Set admin', 'Add minter as admin', and 'Revoke
minter as admin' formed an ordered chain: the latter two acted as
account1 (the admin established by 'Set admin') and reverted when run in
isolation. Merge the three into a single self-contained
'Manages admin and minter roles' test that runs the full narrative from
base state, so it passes whether run alone or with the suite.
The automine consolidation added the .parallel suffix to every converted
test file, but CI runs each it() in a .parallel.test.ts in its own
isolated container (jest --testNamePattern). Files whose it()s share state
or run in sequence (progressive tokenSim mutation validated by afterEach
check(), a describe-scope note/token created in one it() and used by
another, lending positions built across deposit/borrow/repay its) break
when each it() runs alone.

Revert the 12 stateful token files to plain *.test.ts so they run as one
ordered job again: blacklist_access_control, blacklist_shielding,
blacklist_transfer_private, blacklist_transfer_public,
blacklist_unshielding, crowdfunding_and_claim, lending, nft,
transfer_in_public, transfer, transfer_to_private, transfer_to_public.
Files whose its are independent (read-only constants, per-it deploy/mint)
keep .parallel.

Repoint the call_types.md include_code for crowdfunding_and_claim to the
reverted path, and tighten the README .parallel rule: the suffix is for
files with independent its, not merely files with more than one it().
The .parallel split runs each it() via run_test.sh ... "<name>" as a
shell-quoted --testNamePattern regex. An embedded double-quote closes the
quote early and hard-fails the job (pending_note_hashes); regex
metacharacters like parens, brackets, and + silently match zero tests so
the shard runs nothing and passes green, hiding the assertion.

Drop quotes, parens, brackets, and + from the affected it/test names
across the kept .parallel files (pending_note_hashes, 2_pxes,
nested_utility_calls, offchain_effect, phase_check, smoke, access_control,
escrow), keeping them readable. Document the naming constraint in the
README .parallel section.
@spalladino
spalladino merged commit 4151604 into merge-train/spartan-v5 Jun 29, 2026
12 checks passed
@spalladino
spalladino deleted the spl/e2e-automine branch June 29, 2026 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants