Skip to content

fix(video): model-capability guards — root-cause fix for #451/#288 - #537

Merged
ffroliva merged 8 commits into
developfrom
bugfix/451-video-model-capability-guards
Aug 14, 2026
Merged

ffroliva merged 8 commits into
developfrom
bugfix/451-video-model-capability-guards

Conversation

@ffroliva

Copy link
Copy Markdown
Owner

Summary

Flow's video settings popover is model-conditional. Verified live on two accounts and two locales: only omni_flash renders a 4s/6s/8s/10s duration row; the Veo 3.1 models render no duration control at all.

api/video.py claimed "the four VEO_3_1_* models cap at 8s", which presumed a control that is never drawn. _select_video_duration therefore hunted a missing element and died with UiSelectorDriftError (exit 23) after ~30 s.

That explains every confusing symptom on #451/#288: it looked like drift, it reproduced identically on playwright 1.59 and 1.61 (so the version bound was correctly exonerated), and the locale hypothesis was correctly refuted. It was never either.

Changes

  • VideoModel.supports_duration() — true only for OMNI_FLASH.
  • --duration on a Veo model now exits 2 before any browser work, naming the model and the remedy. Guarded at the CLI edge (UsageError, like --ui-mode agentic) because a bare DTO ValueError surfaces as "Unexpected error." exit 1 and the explanation is lost. The DTO guards it too, as defence in depth for API/MCP callers.
  • --reference-entity removed from video i2v, where _validate_i2v_symmetry rejects reference entities — the flag was advertised but raised for every caller who believed it. Still on t2v/r2v.
  • Docs had the mirror-image error: REFERENCE_STRATEGIES.md, USAGE.md and the t2v help text all denied a flag that cli_video.py registered. Corrected in all four places, plus a --duration model-constraint note.
  • ReferenceNotFoundError (exit 32) — --ref-name searches Flow's picker, which indexes Flow's short auto-caption, not the generation prompt. A miss was a bare Playwright TimeoutError after 8 s; it now lists what the picker actually offered.
  • reference_cap_for documents the live-verified ingredient rule it already encoded.

Verified end to end (not just unit-green)

Command Result
t2v --model veo-lite --duration 8 exit 2 + reason (was exit 1 "Unexpected error")
i2v --model veo-fast --duration 6 exit 2 + reason
i2v --model omni-flash --duration 6 passes — negative control, no over-reject
i2v --reference-entity X exit 2 "No such option"
flags t2v ✓ · i2v ✗ · r2v ✓

1689 passed / 3 skipped (cli+api+features) · 162 passed (mcp) · pyright 78 = exact baseline · ruff, hygiene, doc-links, PII, website mirror all green.

e2e caught two defects in this very fix that unit tests and --help both missed: the ValueError surfacing as exit 1, and i2v() crashing with a TypeError because the Click options were removed but the function parameters weren't.

Self-review

/ponytail:ponytail-review on my own diff found supports_image_ingredients() had zero production callers — orphaned when the redundant guard was correctly deleted, since reference_cap_for already encodes the rule. Deleted; evidence folded into reference_cap_for. (351ae6c)

Also included — two #493 negative results

Committed so they are not re-derived, since both hypotheses are compelling:

  1. crop_free trigger — REFUTED. CROP_SELECTORS omits crop_free (the Frames tab icon), and video r2v: UiSelectorDriftError probe=mode_switch_trigger — new Flow editor variant (composer frame slots + Agente toggle) not recognized (v0.55.0) #493's reporter sees Frames-mode slots. Tested on pt-BR: the trigger renders crop_16_9 in both Frames and Ingredients modes. The trigger reflects the aspect, not the sub-mode.
  2. Hydration race — REFUTED by live A/B. A cold editor genuinely has 0 [aria-pressed] nodes and 0 crop_* triggers on load, which appeared to explain both symptoms at once. But with the hydration wait neutered, 3/3 cold loads still succeed: _probe_selector_cascade already waits 4 s per selector for visible. The fix was a no-op, so it was reverted rather than shipped.

Net: #493's editor renders no crop_* across the cascade's full ~24 s budget, so it is a real DOM difference, not timing. Its diag_mode_switch_miss.json remains the gating artifact.

Refs #451, Refs #288, Refs #493

ffroliva and others added 4 commits August 14, 2026 16:55
Flow's video settings popover is MODEL-CONDITIONAL. Verified live on two
accounts and two locales: only omni_flash renders a 4s/6s/8s/10s duration row;
the Veo 3.1 models render no duration control at all.

api/video.py claimed "the four VEO_3_1_* models cap at 8s", which presumed a
control that is never drawn. _select_video_duration therefore hunted a missing
element and died with UiSelectorDriftError (exit 23) after ~30s. That is why the
bug reproduced identically on playwright 1.59 AND 1.61 (the version bound was
correctly exonerated) and why the locale hypothesis was correctly refuted -- it
was never either.

- VideoModel.supports_duration(): true only for OMNI_FLASH.
- --duration with a Veo model now exits 2 BEFORE any browser work, naming the
  model and the remedy. Guarded at the CLI edge (UsageError, like --ui-mode
  agentic) because a bare DTO ValueError surfaces as "Unexpected error." exit 1
  and the explanation is lost -- caught by e2e, not by unit tests.
- The DTO guards it too, as defence in depth for API callers.

--reference-entity was registered on t2v/i2v/r2v, but _validate_i2v_symmetry
rejects reference entities on i2v -- so the i2v form raised for every caller who
believed the help text. Now applied to t2v/r2v only (and i2v's orphaned
parameters removed, which only a real invocation surfaced). The docs had the
mirror-image error: REFERENCE_STRATEGIES.md, USAGE.md and the t2v help text all
denied a flag that cli_video.py registered. Corrected in all four places.

--ref-name searches Flow's picker, which indexes Flow's short auto-caption, not
the generation prompt. A miss surfaced as a bare Playwright TimeoutError after
8s with no exit code to branch on. Now ReferenceNotFoundError (exit 32), which
lists what the picker actually offered.

supports_image_ingredients() documents the R2V ingredient axis (veo-quality
refuses them) by delegating to the existing reference_cap_for table rather than
re-encoding the rule in a second place.

Verified end to end (not just unit-green):
  video t2v  --model veo-lite    --duration 8 -> exit 2 + reason (was exit 1)
  video i2v  --model veo-fast    --duration 6 -> exit 2 + reason
  video i2v  --model omni-flash  --duration 6 -> passes the guard (no over-reject)
  video i2v  --reference-entity X            -> exit 2 "No such option"
  t2v/r2v still expose --reference-entity; i2v no longer does
  1689 passed / 3 skipped across tests/cli tests/api tests/features
  162 passed tests/mcp; pyright 78 = exact baseline; all doc + mirror gates green

Refs #451, Refs #288

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
mode_control.AGENT_TOGGLE_SELECTOR is button[aria-pressed]:has(span.content).
#493 reports an "Agente" pill matching neither it nor the composer-scoped
variant, AND no crop_* settings button. Those read as two separate selector
problems; the capture suggests one cause.

Measured on denon82 (pt-BR, the reporter's locale), $0:

  on load                    aria-pressed nodes = 0   crop_* triggers = 0
  after ensure_media_mode    aria-pressed nodes = 1   crop_* triggers = 1

On load the composer is simply NOT HYDRATED -- neither the pill nor the crop
trigger exists yet. ensure_media_mode succeeds only because it first absorbs the
render race via _composer_present (acted=False is correct here: the pill already
read aria-pressed=false, so no click was needed).

One hydration race explains both #493 symptoms at once, and the exit code too: a
probe that runs before hydration sees nothing, so _detect_non_classic_cohort
matches no agentic indicator and the failure surfaces as exit 23 (drift) rather
than 25 (agentic cohort). A slower client -- the reporter is on macOS -- loses a
race this machine wins.

Captured pill for the record: <button type="button" aria-pressed="false"
class="sc-59223abb-3 dmZGYv"> with a span.content child, i.e. the production
selector DOES match once hydrated.

No product code changed. Next step is to make the mode_switch_trigger probe wait
for composer hydration before declaring drift, rather than adding selectors.

Refs #493

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ve result)

The DOM capture showed a freshly navigated editor has 0 [aria-pressed] nodes and
0 crop_* triggers on load, which looked like it explained both #493 symptoms at
once: no crop_* settings button AND an Agent pill matching no selector, because
at probe time neither exists.

Tested by driving the PRODUCTION _switch_to_video_mode immediately after
navigation, no pre-wait, on denon82 (pt-BR, the reporter's locale), $0:

  with a hydration wait added  : 3/3 cold loads (crop=0, aria_pressed=0) -> ok
  with the wait NEUTERED (A/B) : 3/3 cold loads (crop=0, aria_pressed=0) -> ok

REFUTED. The wait changes nothing, so it was reverted rather than shipped:
_probe_selector_cascade already waits up to 4s per selector for `visible`, which
absorbs the hydration race on its own. Adding another wait is a no-op.

This narrows #493 usefully. The cold-DOM window is NOT the cause, so the
reporter's editor genuinely renders no crop_* trigger across the cascade's full
~24s budget (6 selectors x 4s). That is a real DOM/cohort difference, not a
timing race -- which is why their diag_mode_switch_miss.json is still the
gating artifact.

Second #493 hypothesis killed with evidence (after crop_free in f2f05e6).
Keeping the harness: it A/B-tests any future mode-switch change against a cold
navigation in one command.

Refs #493

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Self-review (ponytail) finding on this branch: supports_image_ingredients() had
ZERO production callers -- only its own definition and three test references. It
became orphaned when the redundant r2v guard was correctly deleted, because
reference_cap_for already encodes the same rule and IS used.

A predicate that exists only to be tested is dead weight, so it is deleted and
its live-verified evidence folded into reference_cap_for's docstring, where the
rule actually lives: a cap of 0 IS the answer to "does this model take image
ingredients?". The test now asserts against that single source of truth.

Verified: 12 passed tests/api/test_video_request.py; pyright 78 = baseline;
ruff clean; hygiene + doc-link + mirror gates green.

Refs #451

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ffroliva
ffroliva marked this pull request as draft August 14, 2026 16:43
/code-review xhigh found 15 issues, including a hole in this fix's own headline
claim and a regression in a shipped scaffold. Fixed, each verified:

HOLE IN THE CORE CLAIM. Both guards returned early when model is None, but the
transport substitutes I2V_DEFAULT_MODEL (veo-lite) for i2v with frames -- so
i2v's DEFAULT path still hit the exact #451/#288 failure. The DTO now resolves
the model the transport will actually use before guarding:

  i2v + frames + --duration, no --model -> REJECT naming 'veo_3_1_lite' (was ACCEPT)
  t2v/r2v with no --model stay unguarded: Flow's sticky default is unknowable here.

REGRESSION IN A SHIPPED SCAFFOLD. `gflow movie template` paired duration with
veo-lite and veo-quality, so the generated starter file raised at request
construction. Scene 1 now uses omni-flash (the only model with a duration
control); scenes 2 and 3 drop duration. Verified: 3/3 template scenes construct
valid requests (2 were raising).

r2v carries --duration but was never wired to the CLI-edge guard, so it fell
through to the DTO's bare ValueError -> "Unexpected error." exit 1. Now exit 2
with the reason, like t2v/i2v.

max_duration_for() still returned 8 for the Veo models, contradicting
supports_duration() twelve lines below and making `gflow models` advertise a
duration users cannot set. Now 0, with the test updated to the verified contract.

All three commands' --help still said "10 requires --model omni-flash", implying
4/6/8 work on Veo models -- the first surface users read, walking them into the
failure. Swept.

ReferenceNotFoundError's message and remediation named a `--ref-name` flag that
does not exist (ref_names is DTO/MCP-only); it was missing from __all__ and from
the USAGE exit-code table; and it had no test. All four fixed. The inline
screenshot clause now reuses the module's screenshot_clause() helper.

_reject_duration_without_control had a dead `is not None` branch masking an
escaping ValueError from from_cli on an unknown alias. Handled; the real
validation reports it (exit 2 with Click's message).

The committed #493 spike docstring still asserted a `mode_control.await_composer_hydrated`
fix that was reverted in the same commit that refuted it -- a future reader would
have concluded a rescue landed that never did. Corrected to record the refutation.

Tests added: exit-32 contract + problem-details (matching the per-error
convention), and a CLI matrix covering t2v/i2v/r2v x veo-lite/fast/quality with
an omni-flash negative control.

Verified: 1684 passed / 3 skipped (api+cli+errors); 10 passed on the new CLI
matrix; 56 passed tests/test_errors.py; pyright 78 = exact baseline; ruff,
format, hygiene (755), doc-links (29), PII, website mirror all green.

Refs #451, Refs #288

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ffroliva
ffroliva marked this pull request as ready for review August 14, 2026 17:04
@ffroliva

Copy link
Copy Markdown
Owner Author

/code-review xhigh findings addressed (eefe2d0)

The review found 15 issues, including a hole in this PR's own headline claim and a regression in a shipped scaffold. Each fix verified:

Hole in the core claim. Both guards returned early when model is None, but the transport substitutes I2V_DEFAULT_MODEL — so i2v's default path still hit the exact #451/#288 failure. The DTO now resolves the model the transport will actually use:

i2v + frames + --duration, no --model  ->  REJECT naming 'veo_3_1_lite'   (was ACCEPT)

t2v/r2v with no --model stay unguarded by design — Flow's sticky UI default is genuinely unknowable at DTO time.

Regression in a shipped scaffold. gflow movie template paired duration with veo-lite and veo-quality, so the generated starter file raised at request construction. Fixed — 3/3 template scenes now construct valid requests (two were raising).

r2v was never wired to the CLI-edge guard, so it fell through to the DTO's bare ValueError → "Unexpected error." exit 1. Now exit 2 with the reason, like t2v/i2v.

max_duration_for() still returned 8 for the Veo models — contradicting supports_duration() twelve lines below, and making gflow models advertise a duration users cannot set. Now 0, test updated to the verified contract.

All three --help strings still said "10 requires --model omni-flash", implying 4/6/8 work on Veo models. That's the first surface users read, and it walked them straight into the failure. Swept.

ReferenceNotFoundError named a --ref-name flag that doesn't exist (ref_names is DTO/MCP-only), was missing from __all__ and the USAGE exit-code table, and had no test. All fixed; the inline screenshot clause now reuses screenshot_clause().

A dead is not None branch was masking an escaping ValueError from from_cli on an unknown alias. Handled.

The #493 spike docstring still asserted a mode_control.await_composer_hydrated fix that was reverted in the very commit that refuted it — corrected, so no future reader concludes a rescue landed that never did.

Tests added

  • exit-32 contract + problem-details, matching the per-error convention in tests/test_errors.py
  • CLI matrix: t2v/i2v/r2v × veo-lite/veo-fast/veo-quality → exit 2 + reason, plus an omni-flash negative control proving no over-reject

Verification

1684 passed / 3 skipped (api+cli+errors) · 10 passed new CLI matrix · 56 passed errors · pyright 78 = exact baseline · ruff, format, hygiene (755), doc-links (29), PII, website mirror — all green.

Still open (not in this PR)

Three review findings deliberately deferred rather than rushed, all pre-existing in shape:

  1. chain.py — a per-link duration aborts mid-run after earlier links are billed, and _resolve_chain_model forces a Veo model, so chain durations are invalid by construction. Needs manifest-level pre-validation before the run starts, which is its own change.
  2. The except PlaywrightTimeoutError around _pick_option_and_include is broader than the name-miss it reports, so an unrelated timeout inside it would be relabelled.
  3. The typed error is at one call site rather than in the shared helper, so _attach_remote_frame's identical i2v miss still raises untyped.

ffroliva and others added 3 commits August 14, 2026 18:19
REPRODUCED the reported fingerprint. Expanding Flow's chat sidebar removes the
classic composer ENTIRELY: no crop_* settings trigger AND no Agent pill. That is
exactly what #493 describes ("no crop_* settings button", "the Agente pill
matches NEITHER selector") -- two symptoms from one state, not two selector
problems. It also explains the exit code: with no agentic indicator on screen,
_detect_non_classic_cohort matches nothing, so the run dies as
UiSelectorDriftError (exit 23) rather than the retryable agentic error (25).

Measured live on denon82 (pt-BR, the reporter's locale), $0:

  in sidebar state -> crop_triggers=0  agent_toggle=0   <- the #493 fingerprint

Recovery hinged on ONE selector, scoped to the sidebar's edit_square ("new
session") affordance:

  div:has(button:has(i.google-symbols:text-is('edit_square')))
  button:has(i.google-symbols:text-is('close'))

That scoping is a single point of failure: a cohort whose sidebar lacks that
ligature never finds the X, the sidebar never closes, the composer never returns,
and every run dies with exit 23.

ensure_media_mode now falls back to an unscoped close, reached ONLY from the
demonstrably stuck state -- no crop_* (loop head) and no Agent pill (call site).
The unscoped match is safe there and nowhere else: the classic composer is gone,
so there is nothing a close button could belong to but the sidebar.

A/B PROVEN live, three configurations:

  scoped OK,     fallback present  -> recovered      (baseline)
  scoped BROKEN, fallback present  -> RECOVERED      (reporter cohort + fix)
  scoped BROKEN, fallback BROKEN   -> NOT recovered  (reporter cohort, no fix)

The middle row is the fix working; the last row proves the fallback is what does
the rescuing, not the surrounding loop.

Unit coverage adds a "sidebar_unscoped" state to the fake page modelling that
cohort, plus a negative control (state="stuck", no close affordance of any shape)
asserting ensure_media_mode still gives up and leaves the caller's probe to fail
loudly -- the documented contract.

Two earlier hypotheses were tested and REFUTED before this one (crop_free
sub-mode trigger, composer hydration race); both negative results are committed
so they are not re-derived.

Verified: 25 passed tests/api/transports/test_mode_control.py; 1631 passed /
3 skipped (api+cli); pyright 78 = exact baseline; ruff, format, hygiene (755),
doc-links (29), website mirror all green.

Closes #493

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI caught what my local scoping missed. tests/test_json_output.py builds its
VideoResult fixture with VEO_3_1_FAST + duration=8 -- a combination the new
capability guard correctly rejects, because the Veo 3.1 models render no
duration control. I ran tests/api, tests/cli, tests/features, tests/mcp and
tests/test_errors.py locally but not this root-level file, so the breakage only
surfaced on the PR matrix.

The fixture now uses OMNI_FLASH, the only model a duration is valid with. The
JSON shape under test is model-agnostic, so the assertion simply follows.

Swept the rest of the tree for the same pattern (a VEO_3_1_* model paired with a
literal duration): tests/api/test_video.py and tests/e2e/test_video_r2v_ref_cap_e2e.py
were flagged and both are clean -- the e2e one references VEO_3_1_LITE only for
reference_cap_for, never a duration. tests/api/test_video_request.py legitimately
carries the pattern because it asserts the rejection.

Verified with the FULL offline suite this time, not a scoped subset:
3115 passed / 5 skipped / 87 deselected. The one other failure in that run
(tests/test_smoke.py::test_imports_succeed) was a stale editable install --
__version__ 0.57.0 vs installed metadata 0.56.0 -- and passes after `uv sync`.
It is environment-only: the diff touches neither that test nor __init__.py, and
CI installs fresh, which is why CI flagged only the fixture above.

Refs #451, Refs #288

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes the three gaps left after the code review.

DOCUMENTATION. KNOWN_ISSUES.md's #493 entry claimed an unrecognized "third
editor variant" awaiting a DOM signature. That framing is wrong and was steering
the issue: "Inicial"/"Final" are the Frames sub-mode's Start/End slots and the
"Agente" pill is the ordinary Agent button -- both stock classic features. The
entry is rewritten Open -> Resolved with the real root cause (the expanded chat
sidebar removes the composer entirely), why the exit code was 23 rather than 25,
why the Portuguese UI was irrelevant, and what to attach if it recurs.

E2E. tests/e2e/test_sidebar_recovery_e2e.py, marked e2e_auth (zero credits: mode
toggles and DOM reads only, no prompt, no Generate). Two scenarios against a real
editor:

  1. the sidebar state really does hide BOTH affordances at once, and production
     recovers from it;
  2. with the edit_square-scoped close selector neutered -- reproducing the
     reporter's cohort on our account -- the unscoped fallback still recovers.

The second is the one that matters: it exercises the fallback for real rather
than assuming it. Unit tests cover the same shapes against a fake page, but only
a live editor proves the sidebar removes both affordances.

The catalog lookup reads the real DB read-only off the profile dir, because the
root _isolate_settings fixture redirects GFLOW_CLI_DB_PATH and sqlite3.connect
would silently CREATE an empty database there -- surfacing as "no such table:
assets" instead of a clean skip.

PONYTAIL CUTS (-330 lines):
- delete scripts/dev/capture_493_hydration_e2e.py and capture_agent_pill_dom.py.
  Both fed the refuted hydration hypothesis; the refutation is recorded in the
  spike doc and the live scenario is now a real e2e. capture_sidebar_state_dom.py
  stays -- it found the answer and carries the A/B controls.
- api/video.py: _has_frame_input was a THIRD copy of the same six-field tuple.
  Now called from _validate_r2v_symmetry and the T2V branch, so the two inline
  copies are gone.

Verified AFTER the review changes, not before:
  live e2e re-run: 2 passed in 34.9s (zero credits)
  full offline suite: 3117 passed / 5 skipped / 89 deselected
  pyright 78 = exact baseline; ruff, format, hygiene (754), doc-links (29),
  PII, website mirror all green

Refs #493, Refs #451

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ffroliva

Copy link
Copy Markdown
Owner Author

Correction to this PR's description. It says "Also included — two #493 negative results". That is wrong for the crop_free one: f2f05e6 lives on #536, not here. This branch was cut from develop before #536 landed.

Accurate split:

Both are green. #536 is evidence-only (scripts/dev/ + spike doc); this PR carries the product changes. They touch different files, so either merge order works.

@ffroliva
ffroliva merged commit 6a3f5ff into develop Aug 14, 2026
14 checks passed
@ffroliva
ffroliva deleted the bugfix/451-video-model-capability-guards branch August 14, 2026 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant