Repository navigation
test: record controller polling delays instead of sleeping them - #886
Merged
Merged
Conversation
SingleControllerExecutor slept each controller's suggested delay (2-30 s) between steps, although every mocked provider answers at once. alien-infra's tests spent ~960 test-seconds idle. - The executor records suggested delays by default and continues; `.real_delays()` keeps real waits for the three live cloud tests. - A suggested delay over 10 minutes fails the run. - The GCP and AWS worker flows assert that every poll waits at least a second, so a hot loop against a cloud API fails a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Assert each phase separately (take_suggested_delays), so a create-phase wait can't cover a delete phase that hot-loops. GCP create and delete, and AWS create, must poll; AWS Lambda deletion completes in the call, so its delete only has to respect the minimum wait. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
Author
|
@greptileai review |
alongubkin
added a commit
that referenced
this pull request
Oct 5, 2026
…896) ## Problem After the recent test fixes, `Rust crates` is ~10.5 min, and the cold build (~5.3 min) is most of it. Every PR builds and tests all 57 packages, even a change to one crate. Replaying the last 50 merged PRs, about half affect 11 or fewer packages. ## Change - **`scripts/affected-rust-packages.py`** reads the PR diff (base..merge commit) and `cargo metadata`. - It selects the changed packages plus every package that depends on them, through normal, dev or build dependencies. - **Any changed file outside the packages selects the full workspace,** unless it is documentation (`docs/`, `*.md`). That covers manifests, the lockfile, toolchain, `.cargo/`, nextest config, workflows, Dockerfiles, scripts, `examples/` and `packages/`. Unknown inputs always mean "test everything". - **Packages under `examples/` and `tests/` are fixtures** that other packages' tests read (the CLI checks every example's `template.toml`), so changes there select the full workspace too. - **A docs-only change selects no packages** (mode `none`), and the Rust job and check are skipped. - **`changes` runs it on pull requests** (stable cargo, so the pinned nightly isn't installed there). Pushes, merge groups and manual runs are always `full`. - **`Rust crates` builds and tests that selection** (`$RUST_SELECTION`), with the CI features limited to the selected packages. - The privileged sandbox-agent tests run when `alien-sandbox-agent` is selected. - The image contract test runs when `alien-core` is selected. - The smoke harness runs in full mode, since its inputs are scripts and Dockerfiles, which always select full. - **New `Rust check` job,** in parallel and only in affected mode: `cargo check --workspace --all-targets` with the CI features, so no package can stop compiling unnoticed. It's added to `All tests passed`. ## Measured (CI benchmark, 2 runs each, `depot-ubuntu-24.04-arm-8`) | PR shape | Full run today | Affected build + tests | `Rust check` (parallel) | |---|---|---|---| | `alien-cli` only (1 package) | ~310 s build + ~214 s tests | 142 s + 2 s | ~193 s | | `alien-infra` (8 packages) | same | 194 s + 44 s | ~187 s | | `alien-core` or `Cargo.lock` (42+) | same | ≈ full | — | The replays select as expected: #819 (CLI only) gets 1 package, #886 (alien-infra) 8, #865 (core) 42. #880 and #891 changed CI files, so they get the full run. This PR changes `ci-fast.yml` and a script, so its own CI takes the full run. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
SingleControllerExecutor(crates/alien-infra/src/core/controller_test.rs) slept for each controller'ssuggested_delaybetween steps. Those are 2–30 s, and a create+delete flow has 10–20 polls. The mocked providers answer at once, so this was pure idle time. GCP worker flows took 37–62 s each, andalien-infraspent ~960 test-seconds per CI run.Nothing checked the delays either. A controller that regressed to polling a cloud API in a hot loop would still pass.
Change
DelayMode::Recordis the default. The executor records each suggested delay (suggested_delays()) and continues..real_delays()keeps real waits for the three#[ignore]live-cloud tests (azure_sandbox_image_live,azure_sandbox_image_refusals_live,kubernetes_sandbox_live).MAX_SUGGESTED_DELAY) fails the run. That catches a seconds/milliseconds mix-up or a runaway backoff.take_suggested_delays,assert_polling_delays).Safe to stop sleeping?
#[cfg(test)]. The seeded-timestamp tests (sandbox build timeout, retire window) seed their times explicitly.httpmockserver, and paused time would let reqwest timeouts fire early.Verified
cargo test -p alien-infra --features all-platforms,kubernetes,local,test --lib --tests: 813 lib tests pass in 11 s, plus the integration tests.alien-infra961 → 199/209 test-seconds, all tests pass.🤖 Generated with Claude Code