Skip to content

I2V Start/End frame-slot selectors still text-based (locale-agnostic follow-up to #24) #63

Closed

Description

@ffroliva

Summary

Spun off from #24 (which is being closed as the umbrella-task scope is otherwise complete). svasakorn flagged this in PR #60's body as scope-keeping:

FRAME_SLOT_BY_LABEL (I2V Start/End slots, div[aria-haspopup='dialog']:has-text('{label}')) is still text-based and will hit the same locale issue on a non-English profile. It's the I2V frame-slot path, separate from this upload fix, and belongs with #24.

Current code

src/gflow_cli/api/transports/ui_automation_video.py:138:

FRAME_SLOT_BY_LABEL = "div[aria-haspopup='dialog']:has-text('{label}')"

Used at line 633 in the I2V Start/End frame-slot binding path. Same root cause as #56: localized text matching (Flow's editor renders labels in the Chrome profile language, not the --lang=en-US Chromium launch arg).

Impact

gflow video i2v on a non-English Chrome profile would hit a silent hang ~38s in (the click-then-no-event signature) when trying to bind a Start or End frame. Same failure mode as PR #60 resolved for _attach_references in i2i.

Suggested fix shape (mirror PR #60)

  1. Investigate the DOM on the affected profile via scripts/dev/capture_image_add_media_dom.py (or a sibling script for the video editor). Need to see whether the Start/End frame slot has a Material Symbols icon ligature (e.g. play_arrow for Start, stop for End, or similar). If yes → use button:has(i.google-symbols:text-is('<ligature>')). If no → structural disambiguation by panel position.
  2. Replace FRAME_SLOT_BY_LABEL with a locale-stable selector or a cascade (icon-first, English-text fallback with short per-tier timeouts so two misses can't recreate a long silent hang — same pattern as PR fix(image): re-introduce i2i ref-attach (#50) with locale-agnostic media-dialog selectors (#56) #60).
  3. Fail-loud if both tiers miss: screenshot + RuntimeError with the explicit "Chrome profile language vs Google account language" hint (PR fix(image): re-introduce i2i ref-attach (#50) with locale-agnostic media-dialog selectors (#56) #60's error message is the model).

Test plan (mirror PR #61)

  • New e2e variant test_e2e_i2v_start_end_frame_attach in tests/e2e/test_transports_e2e.py, GFLOW_CLI_E2E_PROFILE-gated, ui_automation transport ONLY (REST transports drop UI-only fields silently — see rest-transports-drop-ui-fields memory).
  • Assert on a structlog event that ONLY fires when the UI path executed (e.g. start_frame_attached / end_frame_attached if those exist; or add new ones).
  • Verify on a non-EN profile (ffroliva) and an EN profile (svasakorn's setup).

References

Activity

  1. ffroliva commented on May 26, 2026

    @ffroliva
    OwnerAuthor

    Fixed by PR #90.

    PR #70's structural-first cascade was supposed to close this, but the selector SWAP_CONTAINER matched zero elements on real Flow DOMs (the swap_horiz icon uses class material-icons not google-symbols, and the slots are <div type="button"> not children of any div > button > i wrapper). Production I2V silently relied on the EN text fallback and broke on every non-English Chrome profile.

    PR #90 replaces FRAME_SLOTS_STRUCT with the locale-free pattern div[type='button'][aria-haspopup='dialog'] (verified via DOM probe — matches exactly 2 slots), plus fixes a second OOB bug for the End-frame call after Start is attached.

    Live e2e proof: tests/e2e/test_transports_e2e.py::test_e2e_i2v_start_end_frame_attach on ffroliva + GFLOW_CLI_LOCALE=de-DE passed in 124 s — terminal SUCCESSFUL VideoStatus, mp4 downloaded with ftyp magic bytes, both ui_automation_video.frame_attached events fired (Start + End).

  2. added 2 commits that reference this issue on May 26, 2026
  3. added 5 commits that reference this issue on Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions