Skip to content

video i2v silently routes to T2V endpoint — start/end frames dropped, every i2v call burns credits as T2V #125

Closed

Description

@ffroliva

Summary

gflow video i2v silently routes to the T2V endpoint (batchAsyncGenerateVideoText) with startImage: null, endImage: null, and referenceCount: 0 — even when the _attach_frame ↔ _upload_via_open_dialog dance reports clean success for both Start and End slots. Every i2v call burns 1 credit for a pure text-to-video generation; the start/end frames never reach Veo. The result video has zero visual continuity with the ref images and reads as a fresh T2V output.

This was the actual root cause of the wf-004 promo "drift" originally diagnosed (wrongly) as Veo style-shift — _STICKMAN_MOTION ran via T2V the whole time, never as I2V.

Reproduction (gflow v0.10.0, profile promo-denon82, locale pt-BR)

gflow video i2v 01-t2i.jpg "<long prompt>" \
  --end-image 02-i2i.jpg \
  --aspect 9:16 --model omni-flash --duration 8 \
  --profile promo-denon82 --out-dir <tmp>

(01-t2i.jpg and 02-i2i.jpg are 9:16 stickman frames; any pair will reproduce.)

Smoking-gun structured-log sequence

correlation_id b85dde08-5a86-4384-8af5-d4f0ad800298:

ts (UTC) event notes
12:39:17.014 ui_automation_video.image_uploaded target=Start, status=200
12:39:18.976 ui_automation_video.frame_attached slot=Start
12:39:27.573 ui_automation_video.image_uploaded target=End, status=200
12:39:29.434 ui_automation_video.frame_attached slot=End
12:39:29.441 ui_automation.prompt_input_found
12:39:30.140 ui_automation.prompt_submitted via arrow_forward icon
12:39:33.373 ui_automation_video.generate_captured url=.../v1/video:batchAsyncGenerateVideoText

The generate_captured body summary (from _summarize_request_image_inputs):

{
  "parsed": true,
  "startImage": null,
  "endImage": null,
  "referenceCount": 0,
  "referenceIds": []
}

Both uploads + commits reported clean (no upload_incomplete, no dialog_close_timeout). The dialog dance succeeded by every internal signal, but the wire result is a pure T2V request.

Visual evidence

The retry video shows a 2D ink-line stickman in a bedroom at pre-dawn (t=0.1s) and standing beside the bed at sunrise (t=7.5s) — a clean text-driven "wake up" narrative.

The ref images show a stickman at his desk under a lamp (01-t2i.jpg) and arms raised in triumph at sunrise in front of the desk (02-i2i.jpg). Zero overlap. The video reads as pure T2V from the prompt, not as interpolation between the refs.

Local files (not committed):

  • Video: C:\Users\ffrol\gflow-output\i2v-prompt-retry-20260530-133705\retry-i2v.mp4
  • Frames: gflow-cli-remotion/out/promo/v0.10.0-workflow/verify/drift/retry-t{0.1,2,4,6,7.5}s.png
  • 01-t2i / 02-i2i refs: gflow-cli-remotion/public/promo/stickman-001/0{1,2}-*.jpg
  • Full err log: C:\Users\ffrol\gflow-output\i2v-prompt-retry-20260530-133705\_err2.log

Cost impact

Every gflow video i2v call on v0.10.0 currently spends 1 credit for a T2V output. The wf-004 promo run (and possibly earlier sessions that produced "drift" videos) burned credits this way without anyone noticing.

Suspected code location

src/gflow_cli/api/transports/ui_automation_video.py:

  • _upload_via_open_dialog lines 828–842 — the "Add to Prompt" commit step:
    add_btn = (
        page.locator(ADD_TO_PROMPT_DIALOG)
        .last.locator("button")
        .filter(has_not=page.locator("i.google-symbols"))
        .last
    )
    if await add_btn.count():
        await add_btn.click()
    Two silent-failure paths that would not surface in logs:
    1. if await add_btn.count(): await add_btn.click() — if the structural-button-filter matches zero on a non-EN locale, the code silently skips the commit (dialog still closes via outside-click).
    2. The "lone iconless button" heuristic may match a Close/Cancel button instead of "Add to Prompt" on locales where Flow renders the dialog with additional iconless controls. The click would close the dialog (so no dialog_close_timeout) but never commit the image to the slot.
  • The submit (_send_prompt) may be clicking a top-level T2V submit button rather than the frames-panel submit. The prompt_submitted log shows via: button:has(i.google-symbols:text('arrow_forward')) — there may be multiple such buttons in the editor DOM.

Proposed defense-in-depth

Add a wire-level verification step in _attach_frame after _upload_via_open_dialog returns:

  1. Wait for the slot DOM to transition out of the empty div[type='button'][aria-haspopup='dialog'] state (the existing structural anchor). If after ~3s the slot is still empty, raise — the image wasn't actually bound.
  2. OR: probe a known-bound-state attribute (e.g. a data-media-id or thumbnail child) on the slot before declaring success.

A simpler interim guard: add a fail-loud check right before submit that asserts at least one slot is bound when request.mode is Mode.I2V and request.start_image is not None. Currently the code optimistically assumes _attach_frame succeeded if no exception was raised; the structured logs above prove that's not safe.

Environment

  • gflow-cli 0.10.0
  • Profile created via gflow auth login --profile promo-denon82
  • Chrome profile language: pt-BR (Brazilian Portuguese) — Flow's UI rendered "Inicial" / "Final" for the frame slots
  • Windows 11 Pro 26200
  • May relate to #56 (Flow locale leak — icon ligatures) — the same locale-leak class of bug

Activity

  1. ffroliva commented on May 30, 2026

    @ffroliva
    OwnerAuthor

    Root cause confirmed (and corrected) + fix in #128.

    The original "refs never bind / end-image never uploads" hypothesis was wrong — four zero-credit Playwright probes proved the UI attach path is correct (both frames bind; FRAME_SLOTS_STRUCT.count() 2→1→0). There are actually two silent-T2V paths:

    1. omni-flash does not support i2v interpolation. It was the i2v default; Flow's frontend drops the frame refs at submit and routes to batchAsyncGenerateVideoText with image_inputs: null. Proven via page.route(..., abort) interception: omni-flash → T2V/null (both start-only and start+end); veo-lite → batchAsyncGenerateVideoStartAndEndImage with bound mediaIds.
    2. The Veo 3.1 - Lite model-picker selector on develop is broken — exact match :text-is('volume_upVeo 3.1 - Lite') hardcoding a Material Symbols icon prefix that no longer matches. So even an explicit --model veo-lite silently fell back to omni-flash → T2V. (Introduced by feat(video): wire i2v/r2v + model/duration/count picker; fix image --model no-op #48 aa047a3; the avatar branch fixed it but it never merged to develop.)

    Every i2v paid run on v0.10.0 before this fix produced T2V output regardless of the start/end frames — this also retroactively explains the wf-004 promo "drift".

    Fix (#128): drop omni-flash from the i2v --model choices + default to veo-lite; robust :has-text(...):not(...) picker selector; and three guards so i2v can never silently become T2V — a pre-browser DTO guard (ModelModeIncompatibilityError, exit 17), a fatal model-select for i2v (VideoModelSelectionError, exit 18, raised pre-submit so no credit is spent), and a post-submit T2V-routing backstop (WireFormatError).

    Verified live: veo-lite i2v → model_selected → batchAsyncGenerateVideoStartAndEndImage → SUCCESSFUL mp4, a genuine interpolation matching the start/end frames (not a T2V scene-swap).

  2. added a commit that references this issue on May 30, 2026
  3. 10 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions