[ffroliva]

Why

Every gate in clip_qa.py is a per-frame or signal property. On 2026-09-07 a beat whose background monolith tripled in size and materialised top-down passed all of them:

fluid rb04 onset=0.00s face=1.9/0.63 frame=1.88 sync=+0.000s r=-0.11 a/v=+0.000s

fluid, with the second-highest frame-motion of five beats — because a hallucinated object is motion, so it raises the score. The project owner caught it by watching. clip_qa.py's own docstring has said this for months; nothing acted on it.

This is the missing gate: a judge that looks at a 1 fps contact sheet and reports temporal findings.

The invariant

A judge that could not look never returns "nothing found".

Each of these raises JudgeUnavailableError rather than yielding a clean assessment:

Condition	Why it would otherwise read as ACCEPT
finish_reason == "length"	half a JSON object parses leniently to zero findings
unparseable body	ditto
empty body	ditto
verdict=accept with findings	the model contradicted itself; picking a half is guessing
verdict=reject with no findings	nothing to act on
failure name outside the taxonomy	cannot be scored against a fixture
model pool exhausted	silence is not a clean bill of health

This is the same failure this repo has met three times in one week — a 20 s timeout read as "the feature is absent" (#701), a swallowed click read as "the composer is video-only" (#692), an incident bundle photographed after the page was blanked and read as "the DOM was gone" (#722). Silence is not evidence. Here it is a type.

prove_sight() extends clip_qa.py --selftest's discipline to the eye: show the judge a flat colour, refuse to believe any verdict until it names it. The failure it catches is invisible otherwise — a gateway asked for auto may route to a text-only model, which answers plausibly having seen nothing. auto is refused at construction.

Design

The ubiquitous language already existed in skills/video-production/SKILL.md's temporal pass, so this takes it rather than inventing a parallel vocabulary. TemporalFailure mirrors the skill's seven names and a test pins them together — a rename there fails here loudly instead of the two drifting apart.

scripts/dev/acceptance/
  domain.py     TemporalFailure · Verdict · Finding · Assessment · ContactSheet
                FrameJudge (Protocol) · JudgeUnavailableError   — no transport, no config
  settings.py   JudgeSettings — the ONLY module that knows where a judge lives
  judge.py      VisionFrameJudge · ScriptedFrameJudge

No provider name appears in any type. Which gateway, which key, which model family is configuration, resolved from GFLOW_CLI_LLM_* — the same settings the prompt tools and the skillopt harness already use. FrameJudge is a port because there are genuinely two implementations: a model-backed one, and a scripted one so the things that use a judge can be tested offline with a known answer.

Why the judge lives outside clip_qa.py

clip_qa.py states its own contract in its docstring — "ffmpeg and ffprobe only — no model, no numpy, no network" — and that contract is what makes it runnable on any machine, in CI, with nothing installed. A judge is a model call over a network, so it goes in its own module and clip_qa.py stays dependency-free.

The two are complementary rather than competing: clip_qa measures signals, the judge reads pictures. Neither can do the other's job — which is exactly why the monolith defect slipped through carrying a fluid verdict.

Test plan

• 14 new tests, all passing
• Both core invariants mutation-tested — deleting the truncation guard and the self-contradiction guard fails exactly their two tests and nothing else, so neither is a test that would pass against any implementation
• tests/scripts — 257 passed
• pyright 0 errors · ruff clean · ruff format clean
• repo hygiene · doc links · council memory clean

Not verified

No live model call has been made. Every test runs against a fake client. prove_sight() is unit-tested in both directions but has never been pointed at a real gateway, so the network path — real multimodal encoding, a real image_url part, real rotation across a real pool — is unproven. That needs GFLOW_CLI_LLM_* set against a vision-capable endpoint, and it is the first thing to run before this judges anything that matters.

Nothing consumes the module yet, deliberately. Wiring it to the temporal-01 fixture on test/temporal-acceptance-gate is the next step and belongs with that branch.

Refs #685

---

Live-verified since opening

The judge had never made a real call when this PR opened. It has now, and the live path broke four times before it scored anything — each break invisible to a fake client, each now fixed with a regression test naming the live failure:

#	Defect	Why offline testing could not see it
1	prove_sight() wrote into a directory it never created	pytest's tmp_path always exists
2	JSON matched {.*} greedily to the last brace	no fake emitted a stray trailing } — this discarded a correct verdict
3	response.choices[0] on choices=None	a real gateway returned 200 with an error body
4	"no endpoints found" not a dead-model marker	the router's phrasing for all-providers-offline

#2 is the one worth reading. The model answered correctly — right defect, right name, right cell — and closed with one brace too many. Greedy matching swallowed it and reported the answer as unparseable. Refusing to guess is the invariant; refusing to read was a bug. Extraction now counts brace depth, honouring strings and escapes.

The benchmark, live

scripts/dev/acceptance/bench.py scores a judge against a matched pair with known answers — same beat, same cast, same plate, same model, differing only in the defect. Both arms required; a benchmark with only the bad arm is won by rejecting everything, which fails in the direction that looks diligent. The sight proof runs first and is fatal, because a blind judge's two verdicts would look exactly like data.

[ OK ] sight proof passed on nemotron-3-nano-omni-30b-a3b-reasoning:free
[PASS] reject-arm: expected reject, got reject
        background_morph @ cell 3: "a rock appears in the top right corner that is
        not present in cell 0, indicating the background terrain rearranges"
[PASS] accept-arm: expected accept, got accept

BENCH: PASS (2/2 arms correct)

Cell 3 at 1 fps is frame 72 of a 24 fps clip — 3.0s, the exact moment the fragment materialises. The judge found what a human found, and named where.

The "Not verified" caveat above is therefore discharged: the network path, multimodal encoding, rotation and the sight proof have all run against a live endpoint. What remains unproven is breadth — one model, one defect class, one pair.