Skip to content

feat: ship the two demo captures as loadable fixtures - #101

Merged
abrichr merged 1 commit into
mainfrom
feat/legacy-capture-fixtures
Aug 27, 2026
Merged

abrichr merged 1 commit into
mainfrom
feat/legacy-capture-fixtures

Conversation

@abrichr

@abrichr abrichr commented Aug 27, 2026

Copy link
Copy Markdown
Member

What this is for

openadapt-viewer's README embeds six screenshots of the viewer UI (docs/images/*.png, README lines 277–296). A workflow regenerates them when the UI changes and opens a PR, so the documented interface does not drift from the shipped one.

It has produced nothing since 2026-01-19. The committed images are from 2026-01-17; the viewer UI last changed 2026-07-27. The README currently shows a seven-month-old UI.

The workflow reads two demo captures from this repo that were never committed. Committing them as-is would not have fixed it: they predate PR #28 (2026-07-17), which replaced the bespoke capture.db with the SQLAlchemy recording.db that CaptureSession.load reads. Loading either raised FileNotFoundError.

What this adds

scripts/migrate_legacy_capture.py, and the two converted captures under examples/captures/. Each is a single self-contained recording.db — frames live inside as Screenshot.png_data.

capture shows frames actions
turn-off-nightshift macOS System Settings, Night Shift off 20 20
demo_new Spotlight → Calculator, 2 × 3 14 14

The mapping is not guesswork. openadapt_capture/events.py documents it in prose, and the converter follows it exactly:

mouse.move -> move          key.down -> press
mouse.down -> click, pressed=True    key.up -> release
mouse.up   -> click, pressed=False

Actions keep their real timestamps, coordinates and keys. source_ordinal is rebuilt as one sequence over the interleaved frame and action stream, so every action carries a strictly greater ordinal than the frame it binds to — what capture.py requires of a sealed action.

What is reconstructed, stated plainly

Frame timestamps are approximate. A legacy capture kept its frames in video.mp4 and retained only a curated PNG subset; nothing recorded which frame each PNG came from. The converter spreads them evenly across the real recording window. Fine for a fixture and for screenshotting the viewer. examples/captures/README.md says so and says not to measure with them.

One key event in turn-off-nightshift carried no key identity at all. It is dropped, and the converter reports the count. capture.py refuses such an event and is right to; inventing a keystroke would have been worse.

A trap worth knowing about

.gitignore had demo_*/ and turn-off-nightshift/ unanchored. Those match at any depth, so they silently excluded examples/captures/demo_new/ too — this PR would have shipped an empty change. Both are now anchored with a leading slash, which is what they meant. Verified both directions: fixtures are visible, root scratch directories are still ignored.

Privacy

Audio and video are excluded. Both repos are public and the original recordings contain the founder's voice, with one transcript embedded in a generated viewer. create_html(..., include_audio=True) degrades cleanly with no audio, so the generator needs no change.

I reviewed all 34 committed frames. They show macOS System Settings, Spotlight and Calculator. The only identifying string is the shell prompt abrichr@MacBook-Pro-5.

Verification

  • both load via CaptureSession.load
  • every action resolves its bound frame (20 actions over 14 distinct frames; 14 over 14)
  • create_html embeds real JPEG data for every frame, with include_audio both True and False
  • 12 MB total for the pair

🤖 Generated with Claude Code

openadapt-viewer's screenshot workflow regenerates the viewer images in its
README. It has produced nothing since 2026-01-19 because the captures it reads
were never committed, and committing them as-is would not have helped: they
predate PR #28 (2026-07-17), which replaced the bespoke capture.db with the
SQLAlchemy recording.db that CaptureSession.load reads.

Adds scripts/migrate_legacy_capture.py and the two converted captures.

The event mapping is not guesswork. openadapt_capture/events.py documents it:
mouse.move -> move, mouse.down/up -> click with mouse_pressed, key.down -> press,
key.up -> release. Actions keep their real timestamps, coordinates and keys.

source_ordinal is reconstructed as one sequence over the interleaved frame and
action stream, so every action carries a strictly greater ordinal than the frame
it binds to, which is what capture.py requires of a sealed action.

Frame timestamps are reconstructed, not recovered: a legacy capture kept frames
in video.mp4 and retained only a curated PNG subset, and nothing recorded which
frame each PNG came from. They are spread evenly across the real recording
window. examples/captures/README.md says so and says not to measure with them.

One key event in turn-off-nightshift had no key identity and was dropped rather
than given an invented key.

Also anchors two .gitignore patterns to the repository root. Unanchored,
`demo_*/` and `turn-off-nightshift/` match at any depth, so they silently
excluded examples/captures/demo_new/ and would have shipped an empty change.
Root scratch directories are still ignored; verified both directions.

Audio and video are excluded. Both repositories are public and the originals
carry a voice recording; create_html degrades cleanly without audio.

Verified: both load via CaptureSession.load, every action resolves its bound
frame (20 actions over 14 frames, 14 over 14), and create_html embeds real
JPEG frames for all of them with include_audio either True or False.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@abrichr
abrichr force-pushed the feat/legacy-capture-fixtures branch from 818d760 to 7f63e70 Compare August 27, 2026 18:29
@abrichr
abrichr merged commit ad1aaf1 into main Aug 27, 2026
12 checks passed
@abrichr
abrichr deleted the feat/legacy-capture-fixtures branch August 27, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant