Skip to content

test: hermetic regression gate for #147 cache-survives-tar-extract DoD #231

Description

@zackees

Context

Meta #147 (build cache must survive CI tar-extract / cross-runner restore) is functionally complete — all six sub-issues (#146, #148, #149, #150, #151, #152) are closed and the underlying code paths are in tree:

  • crates/fbuild-build/src/build_fingerprint/mod.rshash_watch_set_stamps_inner hashes (len, blake3_content_hash) and uses mtime only as a re-hash skip optimization. relative_path_for_hash strips the watch root from path contributions to hash_files / hash_watch_set / hash_watch_set_stamps.
  • crates/fbuild-build/src/compiler.rs:307build_rebuild_signature hashes compiler_identity(compiler_path) (version + package fingerprint), not the absolute path string.
  • .github/actions/setup/action.yml — emits step output zccache-store-path and exports ZCCACHE_DIR to $GITHUB_ENV.
  • crates/fbuild-build/tests/zccache_hit_across_workspace_rename.rs — proves the zccache wrapper layer produces hits when only the workspace path changes.
  • docs/CI_CACHE.md — consumer-facing recipe.

Problem

#147's Definition of Done lists two empirically-validated criteria — same-runner warm < 10 s and cross-runner warm in the same range — currently covered only by the bench-fastled-examples benchmark (AC#5 ≤ 50 ms warm gate, commit 2e8bc4c).

There is no hermetic regression test that exercises the original failure mode end-to-end:

Save `.fbuild/build/...` to a tar archive, restore it (which resets mtimes and possibly relocates the workspace), then a second build short-circuits on the fingerprint instead of recompiling.

If a future change re-introduces mtime into the watch hash, or bakes an absolute path into `build_fingerprint.json`, the per-layer unit tests would still pass but cross-run CI would silently regress to the iter3 baseline. The benchmark would catch it on the next AC run, but only as a "slow" failure rather than a clear "the fingerprint changed when it should not have" signal.

TDD plan (RED → GREEN)

RED — add `tar_extract_warm_rebuild` integration test

Add `crates/fbuild-build/tests/cache_survives_tar_extract.rs` with three cases:

  1. `fingerprint_survives_tar_roundtrip` — build a small project under `tmpdir/run-a/` via `fbuild-test-support::create_test_project`. Capture `build_fingerprint.json` and the watch-set hash from `hash_watch_set_stamps`. Tar the tree, extract into `tmpdir/run-b/` (canonical mtime-reset op used by `actions/cache`). Use the `tar` crate so the test runs on Windows. Re-compute the watch-set hash from `tmpdir/run-b/`. Assert equality.
  2. `fingerprint_survives_workspace_relocation` — same as above, but extract into `tmpdir/different-parent/run-c/` so the absolute path of the project differs across the two computations. Pins the `relative_path_for_hash` invariant.
  3. `compiler_signature_survives_toolchain_path_change` — construct two compiler inputs with the same version / package fingerprint but different `compiler_path` strings. Assert `build_rebuild_signature(...)` returns equal hashes. Pins the `compiler_identity` substitution at `compiler.rs:307`.

These should pass on `main` today (the fixes are in tree). If they pass, they become the regression gate the meta issue was missing. If any fails, that is the bug — the fix is already supposed to be in tree.

GREEN — if any test fails

  1. Walk `hash_watch_set_stamps_inner` with the failing input and identify which contributor reintroduced mtime / absolute path.
  2. Fix the contributor; do not add a "stable" override that masks the underlying hash. Every input to the hash must be content-derived.
  3. Re-run the suite — including existing fast-path unit tests in `crates/fbuild-build/src/build_fingerprint/fast_path.rs` — to confirm no within-run regression (those tests assume mtime can be trusted within a single daemon lifetime as an optimization signal).

Validation

  • `uv run test -p fbuild-build -- cache_survives_tar_extract`
  • `uv run test -p fbuild-build` — full crate suite
  • `uv run soldr cargo clippy --workspace --all-targets -- -D warnings`

Definition of done

Out of scope

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions