Skip to content

fix(desktop): restore fresh human background screenshots - #22335

Open
Quicksaver wants to merge 1 commit into
pingdotgg:mainfrom
Quicksaver:fix/restore-preview-background-screenshots
Open

Quicksaver wants to merge 1 commit into
pingdotgg:mainfrom
Quicksaver:fix/restore-preview-background-screenshots

Conversation

@Quicksaver

@Quicksaver Quicksaver commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The human Screenshot action can fail to save an image when Electron stops presenting a background preview. Restore the discarded warmup capture, bounded paint-readiness check and independent rendering ownership before the final native readback.

This fresh PR reopens the human fix reverted from #19931, following Julius's re-landing instruction. It restores the original contribution and accepted maintainer adaptations, including independent overlapping captures. Historical Windows, macOS and Linux before/after proof is retained below. #21534 handles agent captures separately.

What changed

  • Keep each human capture's rendering ownership independent of other screenshots, native PiP and recording.
  • Probe paint readiness alongside a discarded warmup, then save the final native image. Existing bounded retries and fallback remain; unconfirmed paint readiness does not guarantee fresh pixels.
  • Restore focused regressions and allow overlapping paint probes to progress independently. Two files, 487 additions and two deletions.

Validation

69 focused tests and scoped checks; historical native comparison

At current HEAD 96715e46db4ed3dbdfd59d1c05002cb5beed2b3f:

  • From apps/desktop: vp test run src/preview/Manager.test.ts src/ipc/methods/preview.test.ts, 69 passing cases.
  • vp run --filter @t3tools/desktop typecheck, touched-file lint and formatting, and git diff --check passed.

Historical native comparison: BEFORE 8eafe058ca7720219e9466e9c6ed8e59a2276dee, AFTER 2e619fae61e66ecd85dd8494a431668920d06af2. All builds, platform observations and images below belong to those revisions. They have not been rerun at current HEAD. Independent source review and the review attribution below also cover that historical AFTER revision.

  • vp run build:desktop passed for both revisions on all three platforms. These are complete unpackaged main/preload/server/web Electron applications, not isolated Manager fixtures or release installers.
  • Windows 11 build 26200 x64, macOS 27.0.1 ARM64, and Debian 12 x86_64 in Docker Desktop WSL2, all using Electron 44.4.5. Linux used Xvfb, Openbox, picom/xrender and llvmpipe software rendering; Playwright supplied --no-sandbox and ephemeral debugging ports. Each app opened genuine existing conversation history imported through the unmodified production scanner/importer before testing, with identical initial state copies and no provider work running.
  • Each acquisition clicked the actual human Capture screenshot control through production preload/IPC/Manager. Independently decoded saved PNG bytes establish the changed epoch and color; DOM observations alone were not treated as image proof.
Observed case Windows BEFORE → AFTER macOS BEFORE → AFTER Linux BEFORE → AFTER
Normal, inactive, minimized Fresh → fresh Fresh → fresh Fresh → fresh
Hidden app window, epoch 4 No PNG → fresh Fresh → fresh No PNG by 12 s → fresh
Two overlapping hidden requests, epoch 8 Neither saved a PNG → two fresh PNGs Two fresh PNGs → two fresh PNGs Neither saved by 12 s → two fresh PNGs
Native PiP active Fresh → fresh Fresh → fresh Fresh → fresh
Close/reopen guest recovery Fresh → fresh Fresh → fresh Fresh → fresh
Display off with hidden target No PNG → fresh, session unlocked Fresh → fresh, session also locked Physical state not tested
Locked, app window visible Fresh → fresh Separate condition not tested Physical state not tested
Recovery after manual unlock Fresh → fresh Fresh → fresh Not applicable

Windows hidden failures took about 3.28 seconds in Manager; one AFTER hidden success took about 53 ms. Linux AFTER saved 8/8 distinct fresh PNGs against BEFORE's 5/8; its 12-second limit was the observer's deadline, not an asserted application timeout. These are observations, not latency guarantees. Windows target-close interruption rejected without an artifact and recovered on a new guest on both revisions. Native PiP/recording ownership released independently; BrowserWindow focus/visibility remained unchanged in the measured background cases. Openbox verified actual Linux minimization.

Limits: Linux establishes the complete app path under a virtual X desktop, not physical monitor sleep/lock, hardware GPU drivers, Wayland or installed AppImage behavior. No natural Mac failure was observed. Mac originals retain their ICC profiles; separate ColorSync sRGB analysis matched the requested colors within one channel value. Mac used an isolated process test-keychain flag, so OS-backed credential sealing is outside this proof. Pending Mac cancellation, a deliberately frozen paint-probe fallback, replacement retaining the exact runtime tab identity, and OS-wide foreground-process preservation were not demonstrated. The human UI stops recording instead of acquiring a concurrent still. An initial AFTER navigation failure and recovery are preserved in the evidence.

Proof

Historical native PNGs, BEFORE failure receipts and reproduction drivers

Windows/macOS proof package and Linux Docker proof package contain the measured cases, saved PNGs, independent pixel results, hashes, limits and reproduction drivers. Both exclude private transcripts, databases and profiles.

Windows hidden app, requested epoch 4:

BEFORE AFTER
No PNG returned. The package retains the failed production request and native window state. Windows AFTER human Screenshot saves current epoch 4 from the hidden app

Linux hidden app, requested epoch 4:

BEFORE AFTER
No PNG within the 12-second observation window. The Linux package retains the request and native window state. Linux AFTER human Screenshot saves current epoch 4 from the hidden app

macOS hidden app, requested epoch 4, no regression observed. These independently acquired originals are byte-identical, so the upload service deduplicated them:

BEFORE AFTER
macOS BEFORE human Screenshot saves current epoch 4 macOS AFTER human Screenshot saves current epoch 4

Every uploaded asset was downloaded again and checked against its original SHA-256; image delivery uses image/png and inline disposition. No baseline failure image was invented when capture returned no PNG.

🤖 Generated by GPT 6.1 Sol in Codex via T3 Code.
🤖 Reviewed until clean by GPT 6 Astra, GPT 6.1 Sol, Claude Fable 5.1, Claude Opus 5.5, CodeRabbit. Findings that reach consensus across the majority were addressed by the generation model.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Oct 20, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This restores fresh background screenshots by adding compositor paint probing, a discarded warmup capture, and independent rendering leases around the existing production screenshot path. Because it changes throttling, debugger coordination, timing, and overlap behavior across screenshot, recording, and picture-in-picture flows, the runtime impact merits human review.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Path: .coderabbit.config.ts
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 61c74c8e-f481-4167-b1c8-e117216880c9
📥 Commits

Reviewing files that changed from the base of the PR and between 2e619fa and 96715e4.

📒 Files selected for processing: 1
  • apps/desktop/src/preview/Manager.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

Still-image screenshot capture now uses a dedicated capture session. It performs a warmup capture while checking paint readiness, then returns a final retried capture. Tests cover readiness, overlap, ownership, failures, guest replacement, and throttling restoration.

Changes: Still-image capture

Layer / File(s) Summary
Capture workflow and lifecycle coverage
apps/desktop/src/preview/Manager.ts, apps/desktop/src/preview/Manager.test.ts
captureScreenshot now calls captureStillImage, which warms the guest and probes for two animation frames in an isolated CDP world. The probe has a one-second timeout, and capture proceeds when readiness is not confirmed. Tests cover retries, pixel selection, overlapping captures, ownership, stale or interrupted captures, guest replacement, debugger handling, and throttling restoration.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant captureScreenshot
  participant captureStillImage
  participant WebContents
  participant IsolatedCDPWorld
  captureScreenshot->>captureStillImage: request still-image capture
  captureStillImage->>WebContents: start warmup capture
  captureStillImage->>IsolatedCDPWorld: probe two animation frames
  IsolatedCDPWorld-->>captureStillImage: report paint readiness or timeout
  WebContents-->>captureStillImage: return warmup result
  captureStillImage->>WebContents: perform final retried capture
  WebContents-->>captureScreenshot: return final pixels
Loading

Suggested reviewers: juliusmarminge

Merge Risk: ⚪ Minimal · up to 96715

This change makes background screenshots more reliable by waiting for paint before the final capture, and the capture still proceeds if readiness cannot be confirmed. No merge-blocking risk is identified from the supplied evidence.

Security architecture risk: 🔵 Low · up to 96715

The change adds local rendering coordination without introducing a new screenshot entrypoint or increasing debugger authority. Independent capture ownership, guest-identity checks and cleanup limit interference with other preview activity. Paint freshness remains best-effort when readiness cannot be confirmed.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The additional runtime exposure is local to the desktop capture lifecycle: the main window's rendering throttling, participating preview guests and their shared recording or picture-in-picture ownership. The inspected change does not add a server capture path, credential access or a new artifact destination.

Trust Boundaries and Controls

  • observed — The readiness probe uses only a control session already owned by the manager; it does not attach a debugger to obtain a screenshot. Existing control-session acquisition refuses DevTools or foreign debugger ownership. Added regressions assert that foreign ownership permits fallback capture without debugger attachment or commands.

Resilience and Maintainability Implications

  • observed — Lifecycle regressions cover overlapping requests, recording and picture-in-picture coexistence, interruption, guest replacement and late warmup completion. Together with consumer-specific release and current-guest checks, these provide source-level counterevidence to premature ownership release or publication of replaced-guest pixels; they are not independent native-platform execution proof.
Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly and concisely describes the primary change: restoring fresh screenshots for hidden human background captures.
Description check Passed The description explains the problem, implementation, scope, approval context, verification results, evidence, and known limitations. It uses custom headings instead of the template headings, but it c…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description explains the problem, implementation, scope, approval context, verification results, evidence, and known limitations. It uses custom headings instead of the template headings, but it contains the required information and is sufficiently complete.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Keep each human capture rendering through a bounded paint probe and
warmup, then save the final native pixels. Let overlapping probes
progress independently while retaining recording and PiP ownership.

Co-authored-by: Julius Marminge <julius0216@outlook.com>

> 🤖 Generated by GPT 6.1 Sol in Codex via T3 Code.
@Quicksaver
Quicksaver force-pushed the fix/restore-preview-background-screenshots branch from 2e619fa to 96715e4 Compare October 21, 2026 09:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant