Skip to content

perf(sessions): store PTY replay buffers as bounded flat pieces - #727

Merged
Juliusolsson05 merged 3 commits into
mainfrom
perf/pty-replay-buffer-chunks
Sep 3, 2026
Merged

Juliusolsson05 merged 3 commits into
mainfrom
perf/pty-replay-buffer-chunks

Conversation

@Juliusolsson05

Copy link
Copy Markdown
Owner

Problem

SessionManager keeps two per-session PTY replay buffers (terminalBuffers, 256 KiB; agentPtyBuffers, 512 KiB) as plain strings and appends with appendCappedBuffer: prev + data, a charCodeAt that flattens the concat into a fresh (cap + chunk) copy, then slice, whose result is a V8 SlicedString keeping that whole parent alive. Once a buffer is full — minutes into any agent session, because TUIs repaint the whole screen — every chunk from every session copies the whole cap on the main thread (with 20 sessions, on the order of 100 MB/s of short-lived allocation, the #390 GC-storm shape), the retained value is cap + chunk rather than cap, and a chunk larger than the cap is retained in full. The 2026-09-03 main-process heap snapshot holds six identical 1.8 MB docker-build repaints and ~10 identical 1 MB Claude screen frames for exactly this reason. Details in #726.

What changed

  • New src/main/sessions/cappedTextBuffer.ts: a list of flat pieces plus a running length. append is amortised O(chunk); eviction drops whole oldest pieces without copying retained bytes; large chunks are cut into surrogate-safe pieces (default 1/16 of the cap, floor 4 KiB) so an overflow discards at most one piece rather than the whole replay; a chunk larger than the cap is replaced by a flat copy of its tail. Pieces are forced flat with the utf16le Buffer round-trip already used in src/main/subagents/shared.ts (fix(#288): make sub-agent body truncation actually free memory (V8 slice trap + toolUseResult miss) #321), so no piece keeps a large parent alive. read() joins once, on attach.
  • SessionManager: the two Map<string, string> become Map<string, CappedTextBuffer>; the five touch points (two inits, two appends, two attach reads) swap over; appendCappedBuffer is deleted. The attach/replay contract — bytes accumulate from process start, attach returns the current tail in the same synchronous block that flips the attached flag — and both caps are unchanged.

Design notes

  • Whole-piece eviction can never split a surrogate pair (chunks arrive as decoded strings; piece boundaries are surrogate-safe), which is why the old low-surrogate check moves into the oversized-tail cut only.
  • The piece size is a constructor parameter so tests can exercise piece boundaries with tiny caps; production uses the default.
  • Dropped pieces are blanked in place and the array is compacted lazily, so a drop is O(1) rather than an O(n) shift per full-buffer append.

Linked issues

Fixes #726. Refs #390 (main-process GC pressure), #321 (sliced-string retention), #103, #327.

Verification

  • NODE_ENV=test npx vitest run --project unit src/main/sessions/ src/main/sessionManager — all passing, including the new cappedTextBuffer.test.ts (retains newest whole pieces; never exceeds the cap with minimal drops across 5,000 random appends; oversized chunk keeps a surrogate-safe tail and later appends continue from it; overflow loss bounded to one piece; no surrogate pair split across pieces; lazy compaction).
  • npx tsc -p tsconfig.node.json --noEmit — clean.
  • After: main eventLoop.meanMs and GC pressure during heavy multi-session output should drop; no session should retain more than its cap.

Plan: docs/superpowers/plans/2026-09-03-pty-replay-buffer-chunks.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kk16MNVJtWAnCeGxGRuHqa

Juliusolsson05 and others added 3 commits September 3, 2026 12:40
Record why the per-session replay strings are replaced by a chunk list
before touching SessionManager: the 2026-09-03 heap snapshot shows the
sliced-string parents the current append keeps alive.

Refs #726

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kk16MNVJtWAnCeGxGRuHqa
SessionManager kept each session's terminal and agent-PTY replay buffer as
one string and appended with `prev + data` → charCodeAt → slice. Once full,
every chunk from every session flattened a fresh cap-sized copy on the main
thread and the stored slice retained that whole parent; a chunk larger than
the cap was retained in full (six identical 1.8 MB docker-build repaints in
the 2026-09-03 heap snapshot).

CappedTextBuffer keeps a list of flat pieces with a running length: append
is amortised O(chunk), eviction drops whole oldest pieces without copying
retained bytes, and a large chunk is cut into surrogate-safe pieces so an
overflow discards at most one piece rather than the whole replay. The join
runs only on attach. The attach/replay contract and both caps are unchanged.

Fixes #726
Refs #390, #321, #103

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kk16MNVJtWAnCeGxGRuHqa
… tests

Review follow-ups: the oversized path copied the tail twice (flatTail, then
cutPieces per piece) — cut straight from the tail offset instead; pieceSize
must be an integer >= 2 so a piece can always hold a surrogate pair, which
makes the boundary pull-back provably non-stalling; the chunk-wholeness
invariant this buffer relies on (node-pty decodes per code point) and the
store-by-reference trade for small chunks are stated in the code. The
surrogate test now uses a chunk whose naive cuts land inside both pairs and
pins the exact retained text per eviction, and a fuzz run with a small piece
size and oversized chunks exercises cutting and the tail path.

Refs #726

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kk16MNVJtWAnCeGxGRuHqa
@Juliusolsson05

Copy link
Copy Markdown
Owner Author

Self-review pass (adversarial review of the first cut) — resolved in HEAD:

  • Surrogate test never executed the pull-back branch → chunk changed to 'ab😀cd😀e' (naive 3-unit cuts land inside both pairs) with exact retained text asserted per eviction.
  • Property test never reached piece cutting → added a fuzz run (cap 1000, pieceSize 37, chunks up to 1,500 units incl. emoji) asserting suffix-of-stream, length ≤ cap, length > cap − pieceSize once over cap, and no low-surrogate head.
  • Oversized path copied twice → cutPieces takes a start offset and cuts straight from the surrogate-safe tail.
  • pieceSize validation → integer ≥ 2, so the fallback branch is gone and the pull-back cannot stall.
  • Invariants stated: chunk wholeness (node-pty StringDecoder) and store-by-reference for small chunks.
  • Compaction test now asserts exact content.

Not adopted: coalescing tiny pieces (spinner case) — overhead measured well under 1 MB; noted as a possible follow-up.

@Juliusolsson05
Juliusolsson05 merged commit b61b0c9 into main Sep 3, 2026
2 checks passed
@Juliusolsson05
Juliusolsson05 deleted the perf/pty-replay-buffer-chunks branch September 3, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(main): PTY replay buffers copy the whole cap on every chunk and retain sliced-string parents

1 participant