[Bug]: Streamed server-browser previews are high-latency and CPU-heavy on remote GPU-less hosts, with no local-render path #16827

Related: #16578 covers image quality and the missing opt-out. This issue covers interaction latency and host CPU, and proposes fixes.

Area

apps/server, apps/web (preview)

Steps to reproduce

1. Run t3 serve in a Linux container with no GPU, limited to a few CPUs, reached from another machine through a reverse proxy and tunnel.
2. Update to 0.0.46-nightly.20261006.2735 or later.
3. Start a dev server in the environment and open it in a preview tab from the web client or the desktop app connected to that remote environment.
4. Scroll, type into an input, and click around.

Expected behavior

Interacting with a preview feels like a local browser, as it did on 0.0.46-nightly.20261005.2702. Back then the client rendered the page itself, and agent automation was available alongside it.

Actual behavior

Since #15328, every preview tab of a remote environment is a JPEG screencast from headless Chromium on the server. Desktop renders natively only for a server it launched itself, so a remote environment is always streamed.

• Latency. Each input does a round trip to the server, waits for the page to repaint, and waits for a frame to come back. The PR's own loopback benchmark shows click-to-frame at p50 76 ms / p90 147 ms. Over a tunnel the network round trip comes on top of that. Typing and scrolling feel noticeably delayed.
• Frame rate. The same benchmark shows 3.3 fps for desktop scrolling and about 12 fps for animation on a 16-core host. Scrolling looks choppy.
• Host CPU. With no GPU, Chrome renders in software. On a 4-core arm64 container (3.5-CPU cgroup quota), with one preview tab open in a client, the headless shell's process tree used about 98% of a core: GPU process ~60%, renderer ~26%, browser ~12%. That CPU competes with the agents and builds running in the same environment.

Impact

Major degradation. In a remote setup, the person works in the preview pane while the agent works in the same environment. Both now share CPU with a software-rendered stream, and the person's interaction lags at every input. There is no setting to restore local rendering.

Version or commit

0.0.46-nightly.20261007.2774 (regression from 0.0.46-nightly.20261005.2702; introduced by #15328)

Environment

Server: Linux arm64 container, no GPU, Debian bookworm, Chrome for Testing headless shell 154.0.8037.92, T3CODE_SERVER_BROWSER_SANDBOX=0. Clients: web client and desktop app connected to the remote environment through an HTTPS reverse proxy.

Possible fixes

Ordered roughly by how much latency each removes:

1. Render locally when the client can reach the page. Let a client (or an environment setting) keep the human-facing tab in a local webview or iframe while agents keep using the server browser. This is the opt-out [Bug]: Server-side browser makes previews blurry, with no way to opt out #16578 asks for. The two can coexist: agent tabs stay headless on the server, and the person's tab renders locally.
2. Serve environment ports through the server. A preview gateway that proxies localhost:<port> over the environment's authenticated HTTP endpoint would let a remote client render a dev server locally with no streaming. Earlier releases refused the environment-port target for an environment the client could not reach, and the error named a preview gateway that hadn't shipped yet. Combined with (1), this covers remote setups without a VPN.
3. If streaming stays the default, make it cheaper and smoother.
   • Use a video codec (H.264/VP8 over WebRTC or WebCodecs) instead of per-frame JPEG. That lowers bandwidth and makes higher frame rates affordable.
   • Let clients pick quality and frame rate. /api/preview-stream/ws already takes quality, but clients never send it.
   • Keep the motion-mode downscale optional.
4. Document the cost. State that each actively viewed streamed tab costs roughly one core on GPU-less hosts, so people can size environments, or turn streaming off for human viewing.

Workaround

• Open the dev server's URL in a normal browser tab, using any route that reaches the environment (reverse proxy, VPN). Agents still use the server browser.
• Or pin the server to 0.0.46-nightly.20261005.2702, which loses the newer nightlies.