perf(server): a returning client catches up in batches, not one event per round trip

[tusharbhardwaj-bk]

Problem. When a window comes back and resumes a thread, the server replays the events it missed. The replay pipeline taps each event to notice a delete or archive. An effectful Stream.tap re-emits one event per chunk, and the RPC server writes each chunk as its own WebSocket frame, then waits for the client's ack before writing the next. Catch-up time therefore scaled as missed events × round trip.

Measured on expbkt3 from the server host: resuming 500 events behind arrived as 493 frames, one per event. From Tushar's laptop in India to ap-south-1, that is 500 serial round trips before the thread is current, plus the client's per-frame handling. The server's own work is milliseconds.

Measured cost before the fix (expbkt3, a real 560 KB thread; test client holds each ack for a simulated round trip):

Missed events	0 ms round trip	35 ms round trip (India → ap-south-1)
50	46 ms, 51 frames	1.8 s
500	0.4 s, 493 frames	17.9 s

With 128-event batches, 500 events are 4 frames, so the same catch-up is about 4 round trips plus transfer.

Fix. Re-chunk the catch-up replay into batches of 128 (threadReplayBatches.expbkt3.ts, one marked line in ws.ts). The replay is a bounded read that already arrives page by page, so filling a batch adds no latency. The client already applies multi-item chunks as one batch (applyItems in client-runtime/src/state/threads.ts), so 500 missed events become 4 frames. The live stream is untouched; its coalescer already batches.

Tests.

• threadReplayBatches.expbkt3.test.ts reproduces the real pipeline shape (paginated read → tap → filter → map). Unbatched, it reaches the writer one event per frame; batched, it arrives in 128-event frames, in order, and a short replay is flushed at once.
• The existing subscribeThread server tests pass (5 of 5).

This is generally useful, so it's an upstream candidate.

🤖 Generated with Claude Code

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	—	13.9 KiB	—	15.1 KiB	✅
Codex	Thread snapshot wire	—	7.3 KiB	—	7.8 KiB	✅
Codex	Live turn WebSocket wire	—	6.5 KiB	—	7.8 KiB	✅
Codex	Live turn WebSocket decoded	—	57.1 KiB	—	66.4 KiB	✅
Codex	Live turn messages	—	10	—	21	✅
Claude	Total thread wire	—	13.8 KiB	—	15.1 KiB	✅
Claude	Thread snapshot wire	—	7.3 KiB	—	7.8 KiB	✅
Claude	Live turn WebSocket wire	—	6.5 KiB	—	7.8 KiB	✅
Claude	Live turn WebSocket decoded	—	57.9 KiB	—	66.4 KiB	✅
Claude	Live turn messages	—	9	—	21	✅

Baseline: unavailable · PR result: d5e188a · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 114.9 KiB
• Claude decoded thread snapshot: 115.6 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.