Repository navigation
test: run the worker integration tests in their own job, as two real tests - #891
Merged
Merged
Conversation
…tests The 11 alien-local worker tests set the Rust job's wall clock: the first one built the bindings addon in release mode while the rest waited, then each of 8 built the example app again, one at a time (~600 test-seconds per run). - A path-filtered `Worker integration` job builds the addon and AI gateway once (debug) and runs the suite; the main Rust job filters it out. - The 8 build-heavy tests become 2: one worker through its whole lifecycle, and two workers from one image started side by side. - Dropped `test_delete_prevents_port_reuse` (could fail on correct code) and `test_port_allocation_fallback` (asserted nothing beyond restart). - A missing Bun now fails the tests instead of skipping them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
Author
|
@greptileai review |
alongubkin
added a commit
that referenced
this pull request
Oct 5, 2026
…894) ## Problem The `alien-worker-runtime::container_app` tests (5 tests, ~236–341 test-seconds in the Rust job) each start a real `alien-test-app` process. - `test_utils::get_test_app_path` builds it with `cargo build -p alien-test-app` and caches the path in a `Once`. But nextest runs each test in its own process, so every test runs that build. - `-p alien-test-app` resolves features differently from the workspace test build. So the first build recompiles dependencies, and the others wait on the target-dir lock. Each test takes ~47 s, nearly all of it waiting. ## Change - **`get_test_app_path` uses `ALIEN_TEST_APP_BIN` when it's set,** and fails if that path isn't a file. Without it, the tests build the binary themselves as before, so local `cargo test` is unchanged. - **The `Worker integration` job (#891) builds `alien-test-app` once,** in the same `cargo build` as the addon and gateway. It exports `ALIEN_TEST_APP_BIN` and runs `container_app` together with `worker_integration`. - **The main Rust job filters `container_app` out.** The test binary still compiles there. - **The job now runs on any change matching the `rust` filter,** plus the JS inputs in `worker_integration`. The tested packages, alien-test-app, the addon and the gateway depend on 31 workspace crate directories, so a hand-kept path list would drift. It runs in parallel, so it adds no wall-clock time. - **It runs only the two test targets** (`--test worker_integration --test container_app`). ## Verified locally With `ALIEN_TEST_APP_BIN` set: 5 passed, 3–6 s each. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
alongubkin
added a commit
that referenced
this pull request
Oct 5, 2026
…896) ## Problem After the recent test fixes, `Rust crates` is ~10.5 min, and the cold build (~5.3 min) is most of it. Every PR builds and tests all 57 packages, even a change to one crate. Replaying the last 50 merged PRs, about half affect 11 or fewer packages. ## Change - **`scripts/affected-rust-packages.py`** reads the PR diff (base..merge commit) and `cargo metadata`. - It selects the changed packages plus every package that depends on them, through normal, dev or build dependencies. - **Any changed file outside the packages selects the full workspace,** unless it is documentation (`docs/`, `*.md`). That covers manifests, the lockfile, toolchain, `.cargo/`, nextest config, workflows, Dockerfiles, scripts, `examples/` and `packages/`. Unknown inputs always mean "test everything". - **Packages under `examples/` and `tests/` are fixtures** that other packages' tests read (the CLI checks every example's `template.toml`), so changes there select the full workspace too. - **A docs-only change selects no packages** (mode `none`), and the Rust job and check are skipped. - **`changes` runs it on pull requests** (stable cargo, so the pinned nightly isn't installed there). Pushes, merge groups and manual runs are always `full`. - **`Rust crates` builds and tests that selection** (`$RUST_SELECTION`), with the CI features limited to the selected packages. - The privileged sandbox-agent tests run when `alien-sandbox-agent` is selected. - The image contract test runs when `alien-core` is selected. - The smoke harness runs in full mode, since its inputs are scripts and Dockerfiles, which always select full. - **New `Rust check` job,** in parallel and only in affected mode: `cargo check --workspace --all-targets` with the CI features, so no package can stop compiling unnoticed. It's added to `All tests passed`. ## Measured (CI benchmark, 2 runs each, `depot-ubuntu-24.04-arm-8`) | PR shape | Full run today | Affected build + tests | `Rust check` (parallel) | |---|---|---|---| | `alien-cli` only (1 package) | ~310 s build + ~214 s tests | 142 s + 2 s | ~193 s | | `alien-infra` (8 packages) | same | 194 s + 44 s | ~187 s | | `alien-core` or `Cargo.lock` (42+) | same | ≈ full | — | The replays select as expected: #819 (CLI only) gets 1 package, #886 (alien-infra) 8, #865 (core) 42. #880 and #891 changed CI files, so they get the full run. This PR changes `ci-fast.yml` and a script, so its own CI takes the full run. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
alien-local::worker_integrationset theRust cratesjob's wall clock. Onmaintoday it takes 607 test-seconds of a 607 s test run.napi build --release, ~5 min), and the other tests waited on its lock.alien buildofexamples/basic-worker-tsagain, one at a time.In 60 days its failures were all readiness timeouts, install errors or build races it caused itself. None was a product bug.
Change
Worker integrationjob inci-fast.yml, in parallel withRust crates.cargo build -p alien-bindings-node --lib, debug, staged asalien-bindings-node.linux-arm64-gnu.node) and the AI gateway once.cargo nextest run -p alien-local --test worker_integration.All tests passed.max-threads = 1: twoalien buildruns in the same example directory could delete each other's generated files.worker_lifecycle_on_one_imagechecks:Localbinding URL equals the worker URL/healthreturnsokwith a timestamp, and the health check passestwo_workers_from_one_image_run_side_by_side: two workers from one image start at the same time, get distinct ports, and both serve/health. Both run the same Bun-compiled binary, so this also covers processes sharing its embedded addon.test_delete_prevents_port_reuse: it asserted two random ports differ, so it can fail on correct code.test_port_allocation_fallback: it only printed which port it got.bunnow fails the tests instead of skipping them.Verified locally (macOS, addon prebuilt)
cargo nextest run -p alien-local --test worker_integration: 5 passed in 61 s.🤖 Generated with Claude Code