Skip to content

The verdict cache remembers which switches the pass ran under - #382

Merged
blooop merged 1 commit into
mainfrom
fix/verdict-cache-remembers-its-switches
Aug 24, 2026
Merged

blooop merged 1 commit into
mainfrom
fix/verdict-cache-remembers-its-switches

Conversation

@blooop

@blooop blooop commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Found by a retrospective wf-review of #328, which merged two minutes after opening on CI-green auto-merge. Sourcery left high-level feedback; nothing looked at the cache's correctness.

The defect

Marker recorded {verdict, result_mtime} — which container the pass was about, and nothing about what that pass carried.

So:

  1. DEVLAUNCH_NO_ZELLIJ=1 dl <ws> runs a setup pass with no zellij stage.
  2. That pass probes provisioned — correctly. The probe asks whether the tools answer; zellij is not what it is about.
  3. A marker is written: this container, provisioned.
  4. A later dl <ws> with no variable set wants the zellij stage. It finds a trusted marker for a container that never stopped, returns CachedProvisioned, and skips the trip.

zellij is never installed. And it never will be: every subsequent top-up reads that same marker and skips that same trip, so the failure is permanent for the life of the container and completely silent. The container has to be recreated before anything notices — which is the same shape as the herdr feature withdrawn in #376, a feature whose majority case was "installs nothing" with nothing saying so.

The marker answered "was this container provisioned?" when the question the flow asks it is "was it provisioned the way this launch wants?" Those differ exactly when somebody used an opt-out, which is the only reason the opt-outs exist.

The fix

Marker gains switches, and trusted() takes the switches this launch wants. A marker written under different switches reads as no verdict.

The comparison runs before the anchor check: it is cheaper, and it is the one that is wrong about a container that really is standing — the mtime can match perfectly while the marker describes a pass that skipped the stage being asked for.

MarkerSwitches is two bools in a private struct rather than serde derives on the pub Switches. Deriving on the public type would put the on-disk format of a cache file into the frozen surface, where a field rename becomes a breaking change to something no consumer ever asked for.

Existing markers

A marker written by 0.12.0 or earlier has no switches key, so it fails to deserialize and reads as absent — one redundant round trip on the first launch after upgrading. That is the direction Verdict's own doc argues is the only harmless one, and it is why the enum-with-one-arm trick was chosen there in the first place. Treating a keyless marker as a full install would silently re-open the defect for every container that already exists.

Verified

  • Red before green. a_marker_written_by_an_opted_out_pass_is_not_trusted_by_a_full_launch fails with the switch comparison removed and passes with it. It asserts both directions: the pass that wrote the marker must still be answered by it, and a launch wanting the zellij stage must not be.
  • a_marker_from_a_build_that_did_not_record_switches_is_not_trusted strips the key from a real marker on disk and checks the reading.
  • cargo test --workspace: 1,673 passed, 0 failed. cargo clippy --locked --all-targets -- -D warnings clean, cargo fmt --check clean.
  • No public-API snapshot movement: Marker and MarkerSwitches are private, and trusted/record are pub(crate).

From the same review, not fixed here

  • DEVLAUNCH_NO_TOOLS=1 on a cached top-up returns CachedProvisioned instead of Disabled, dropping the ProvisioningDisabled event and flipping tools_present() false→true. The cache is consulted above the ToolsSwitch::Skip arm. Related, but a different ordering fix.
  • Three atomic-write implementations (metadata.rs:596, completion_cache.rs:165, verdict_cache.rs:243) that disagree on fsync, interrupt-time cleanup and error policy — the newest is the only one that swallows every error and never removes its staged temp. This is Sourcery's Two ways to stop paying for the setup pass #328 finding, confirmed and drifted in the three days since; it belongs under A standing rule for second copies of one fact #319.
  • trusted answers "is this the same container", which the flow reads as "are the tools still there" — tools removed from a container that never stopped are never noticed. Undocumented residual.
  • A surviving mutant: replacing the anchor equality with a seconds-only comparison leaves all 1,249 core tests green, so the nanosecond half of the same-second-collision defence is untested.

Full report on #328.

🤖 Generated with Claude Code

Summary by Sourcery

Ensure cached provisioning is reused only when the cached pass matches the switches requested by the current launch.

Bug Fixes:

  • Prevent verdict-cache entries created with provisioning opt-outs from being trusted by launches that require the skipped stages.
  • Treat markers from older builds without recorded switch settings as untrusted so cached provisioning cannot silently bypass required work.

Enhancements:

  • Make cached provisioning verdicts depend on the launch switches used to create them.

Documentation:

  • Document the cache correctness fix and its behavior for markers written by earlier builds.

Tests:

  • Add coverage for switch-mismatched markers and legacy markers without switch metadata.

A marker recorded the container a pass was about and nothing about what
that pass carried, so a DEVLAUNCH_NO_ZELLIJ=1 pass -- which probes
provisioned, because the probe is about the tools -- wrote a marker the
next full launch trusted. zellij was never installed, and no top-up
could ever notice: they all read the same marker and skip the same trip.

MarkerSwitches is two bools in a private struct rather than serde on
the pub Switches, so the on-disk format of a cache file stays out of
the frozen surface.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @blooop, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@sourcery-ai

sourcery-ai Bot commented Aug 24, 2026

Copy link
Copy Markdown

Reviewer's Guide

The verdict cache now records the tools/zellij switches used by each provisioning pass and only trusts markers for launches with the same switch configuration, preventing an opt-out pass from permanently suppressing a later required stage while safely invalidating legacy markers.

Sequence diagram for switch-aware verdict cache validation

sequenceDiagram
    participant Launch
    participant Provision
    participant VerdictCache
    participant Container

    Launch->>Provision: provision(workspace, switches)
    Provision->>VerdictCache: trusted(workspace, switches)
    VerdictCache->>VerdictCache: read_marker()
    alt marker switches match launch
        VerdictCache->>Container: sole_workspace_result()
        Container-->>VerdictCache: matching result mtime
        VerdictCache-->>Provision: true
        Provision-->>Launch: CachedProvisioned
    else switches differ or marker is legacy
        VerdictCache-->>Provision: false
        Provision->>Container: run provisioning stages
        Container-->>Provision: observed result
        Provision->>VerdictCache: record(workspace, observed, switches)
    end
Loading

Entity relationship diagram for switch-aware verdict markers

erDiagram
    VERDICT_CACHE ||--o| MARKER : stores
    MARKER {
        Verdict verdict
        Stamp result_mtime
        MarkerSwitches switches
    }
    MARKER_SWITCHES {
        bool tools
        bool zellij
    }
    MARKER }o--|| MARKER_SWITCHES : records
Loading

File-Level Changes

Change Details Files
Make verdict-cache entries specific to the provisioning switches used to create them.
  • Add a private serialized switch fingerprint for tools and zellij install/skip modes.
  • Require the current launch switches to match before trusting a cached verdict, checking this before container-anchor validation.
  • Record the switch fingerprint alongside newly observed provisioned verdicts.
rust/devlaunch-core/src/flows/provision.rs
rust/devlaunch-core/src/flows/provision/verdict_cache.rs
Prevent stale or legacy markers from incorrectly satisfying current provisioning requests.
  • Treat markers without the new switches field as unreadable and therefore untrusted.
  • Add regression coverage for opted-out passes followed by full launches and for pre-switch marker files.
  • Update existing cache tests to pass and validate launch switches.
rust/devlaunch-core/src/flows/provision/verdict_cache.rs
Document the cache-correctness fix and its upgrade behavior.
  • Describe the permanent silent zellij-skipping failure and switch-aware invalidation in the unreleased changelog.
  • Document why the private on-disk representation and fail-closed legacy behavior preserve the public API and safe compatibility direction.
CHANGELOG.md
rust/devlaunch-core/src/flows/provision/verdict_cache.rs

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@codecov

codecov Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.20%. Comparing base (a5f7ed8) to head (8a682c0).

Additional details and impacted files
Flag Coverage Δ
python 42.98% <ø> (ø)
rust 95.56% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
shipped code (rust) 95.56% <100.00%> (+<0.01%) ⬆️
harness and tooling (python) 42.98% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@blooop
blooop merged commit 943ec2d into main Aug 24, 2026
14 checks passed
@blooop
blooop deleted the fix/verdict-cache-remembers-its-switches branch August 24, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant