Repository navigation
[Demos] Refactor+Move waypoint into v2 - #579
Conversation
Greptile SummaryThe PR introduces a reusable Action2V application, ports Waypoint to the v2 integration API, and adds runtime cursor visibility and capture controls.
Confidence Score: 4/5The PR is not yet safe to merge because the previously reported promptless MP4 path still completes without generating video. A missing prompt is still passed into Files Needing Attention: apps/t2v/t2v/application.py, apps/t2v/t2v/session.py Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant Action2V
participant SessionRunner
participant UILoop
participant Window
participant Waypoint
User->>Action2V: Start application with image and controls
Action2V->>Waypoint: Resolve seed and construct adapter
Action2V->>SessionRunner: Create Action2V session
SessionRunner->>UILoop: Initialize UI loop
UILoop->>Window: Request hidden, locked cursor
Window-->>SessionRunner: Keyboard and mouse events
SessionRunner->>Waypoint: Map actions and generate frames
Waypoint-->>SessionRunner: Video chunk
SessionRunner-->>Window: Present generated frames
Reviews (5): Last reviewed commit: "clean up and address review" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
2be6b96 to
20ba4b1
Compare
|
/ok to test 20ba4b1 |
ArielG-NV
changed the title
|
/ok to test f1eef2a |
| raise ValueError("--mouse-sensitivity must be finite and non-negative") | ||
| self._validate_arguments(args) | ||
|
|
||
| seed = secrets.randbits(63) if args.seed is None else args.seed |
There was a problem hiding this comment.
if we use random seed by default, make sure the benchmark/CI is aware of that and provides the seed arg
There was a problem hiding this comment.
I will ensure use matches cam2v.
|
|
||
| def is_finished(self) -> bool: | ||
| """Keep the live model loop running until the runtime stops it.""" | ||
| return False |
There was a problem hiding this comment.
[P1] Public MP4/null launches have no finite completion or deterministic controls
Action2VModelLoop.is_finished() always returns False, while the public Action2V app accepts neither a control file nor an action limit. The CLI only exposes a wall-clock timeout, not the internal run_session(..., steps=...) bound used by tests. As a result, normal MP4/null execution cannot finish deterministically or replay a specified control trace, and the packaged example_controls.json is unreachable from the production path. This misses the bounded finite-replay requirement in #479. Please add a finite control source/action limit, finish when it is exhausted, and cover the public ApplicationRunner/CLI path.
There was a problem hiding this comment.
Note: comment was from Sol, I reviewed it, I need to modify skills/settings so my agents always attribute .
f1eef2a to
168014a
Compare
|
/ok to test 75907e9 |
Related to larger porting effort: #479
Changes:
Adds a reusable
action2vdemo application for interactive action-conditioned video models. It owns the shared session loop, input mapping, and (mouse) cursor setup so integrations only need to provide their model-specific adapter. This is just initial support, extensive controller support (etc...) can be managed later.Ports Waypoint to the v2 API using the new Action2V application. The Waypoint implementation, configuration, tests, and example controls are moved into the
integrations_v2/waypointdir.Extends the
IClientWindowandIUILoopprotocols with support forhide_cursorandlock_cursor_to_window. UILoop requests are now bundled all in one object calledUILoopRequest.Adds runtime cursor control for native and WebRTC windows. Action2V requests a hidden cursor locked to the window through
invoke_async. Default behavior is a visible not-locked-to-window cursor. WebRTC sends the initial cursor state when a client connects and only sends further updates when the effective options change.