Observed on PR #465, 2026-08-06. Two zig CI runs against the identical head SHA (0fea4f3): one failed, one passed.
tournament E2E failed; fixture preserved at /tmp/codegraff-learn-tournament.ob35k_n2
assert (mutation_failure_barrier / "mutator-attempts-1").read_text() == "2"
FileNotFoundError: [Errno 2] No such file or directory:
'/tmp/codegraff-learn-tournament.<tmp>/barrier-mutation-failure/mutator-attempts-1'
Failing run: https://github.com/justrach/codegraff/actions/runs/31104029504/job/92624467497
Passing run, same SHA: https://github.com/justrach/codegraff/actions/runs/31104035680
Same commit, opposite outcomes, so this is genuinely non-deterministic rather than a content regression. The PR that surfaced it (#418) changed only a new test file, test_hooks.zig and comments, none of which the tournament path touches.
What the assertion wants: the barrier fixture expects the mutator to have been attempted twice, recorded as a counter file. The file is absent entirely, not holding a wrong value — so the failure is upstream of the count: either the barrier directory was never populated, or the assertion ran before the writer finished.
Given #444 established that this repo's other long-standing "flake" was a misread of build-runner output rather than real non-determinism, it is worth resisting the same assumption here. This one has hard evidence of true non-determinism (identical SHA, both outcomes), so the mechanism is real and worth finding rather than papering over with a retry.
Likely candidates, in the order worth checking:
- A write/read race — the harness asserts on a counter file the mutator subprocess writes, without waiting for it. A missing file rather than a stale value fits this best.
- Fixture teardown or
/tmp reuse — the fixture path is preserved on failure, so a preserved-but-empty directory from the failing run would distinguish this quickly.
- A spawn that failed silently, leaving the barrier unpopulated, with the assertion reporting the symptom rather than the cause.
Whoever takes this: the preserved fixture path in the failure output is the first thing to look at, and the two runs above give a working pair to diff.
Observed on PR #465, 2026-08-06. Two
zigCI runs against the identical head SHA (0fea4f3): one failed, one passed.Failing run: https://github.com/justrach/codegraff/actions/runs/31104029504/job/92624467497
Passing run, same SHA: https://github.com/justrach/codegraff/actions/runs/31104035680
Same commit, opposite outcomes, so this is genuinely non-deterministic rather than a content regression. The PR that surfaced it (#418) changed only a new test file,
test_hooks.zigand comments, none of which the tournament path touches.What the assertion wants: the barrier fixture expects the mutator to have been attempted twice, recorded as a counter file. The file is absent entirely, not holding a wrong value — so the failure is upstream of the count: either the barrier directory was never populated, or the assertion ran before the writer finished.
Given #444 established that this repo's other long-standing "flake" was a misread of build-runner output rather than real non-determinism, it is worth resisting the same assumption here. This one has hard evidence of true non-determinism (identical SHA, both outcomes), so the mechanism is real and worth finding rather than papering over with a retry.
Likely candidates, in the order worth checking:
/tmpreuse — the fixture path is preserved on failure, so a preserved-but-empty directory from the failing run would distinguish this quickly.Whoever takes this: the preserved fixture path in the failure output is the first thing to look at, and the two runs above give a working pair to diff.