perf(sessions): read only the transcript tail when loading history

[Juliusolsson05]

Problem

Loading a resumed session's history parsed the WHOLE transcript to return its newest 120 records: readInitialTranscriptTail streamed every line through readline + JSON.parse, interned it, and kept a ring buffer of limit; readOlderTranscriptWindow forward-scanned from byte 0 to the renderer's anchor on every page.

Measured in #747 (today's boot): 25 historyLoader.loadInitialChunk spans read 252 MB in 10.7 s of main-thread time. The worst — an 89.3 MB / 18,366-line rollout — took 4,618 ms to return 120 entries (332 ms even hot from page cache). All 25 ran in the first 8 s of boot, where main's event-loop p99 hit 577 ms and nine parallel provider spawns each showed 950–2,130 ms of non-provider time; renderer workspace.history.loadInitial totalled 38.7 s behind INITIAL_HISTORY_CONCURRENCY = 2.

Behaviour

• The initial load reads 256 KiB blocks backwards from EOF, splits on newline bytes, and parses newest-first until limit + 1 usable records are found. The extra record is what hasMore means (at least one more usable record exists), exactly as parsed > limit meant before. Parse cost is proportional to the window.
• totalEntries = records parsed in the tail + one no-decode newline count over the untouched head (1 MiB blocks). This is still a pass over the head's bytes (17–32 ms hot on 84 MB; a full-file read cold) — the load's I/O is not independent of file size, its parse cost is.
• Every chunk carries offsets — the absolute byte offset of each returned record's line, parallel to entries. The renderer keeps the offset of its cursor line (historyOldestOffset, next to historyOldestMarker) and sends it back as beforeOffset.
• The older page with beforeOffset: the loader verifies the line at that offset carries beforeMarker (one record read, and the byte before it must be a newline), then walks backward from exactly that byte for limit + 1 records — exact, O(page), no marker hunt. Without an offset (remote client; a cursor the live-window trim re-anchored on an entry with no recorded position) or when the check fails (rewritten file, cursor from another transcript), it runs the original forward scan anchored on the oldest occurrence of the marker, which parses from byte 0 to the anchor as before this PR and keeps the "marker not in file → page from the tail" fallback. That page returns offsets, so the very next page is exact again.
• Records containing a raw U+2028/U+2029 (valid JSON; Codex writes them) are one line. Node readline treated those as line terminators, so the old reader shredded such records into junk fragments; they now parse intact.
• Unchanged: which records are returned and in what order, parse-error tolerance (blank lines skipped, malformed lines counted and skipped, unterminated final line treated as a line), the per-load intern pool (Main-process heap unbounded — full transcript + tool-output text retained in memory (681MB, 92% strings) #288, now only for returned entries), hasMore semantics, the empty-chunk read-failure path. The IPC and remote wire shapes gain optional fields only (offsets on chunks, beforeOffset on older requests).

Design decisions

• Why a position cursor, not a marker hunt (review blocker): markers are not unique. Claude transcripts on the reviewing machine held 565 duplicated uuids across 4 of the 60 largest files, in contiguous blocks of 100–260 records repeated 200–530 records later with different content; Codex synthesized markers collide routinely (6,277 duplicates in 49/60 rollouts, mostly same-millisecond ts:type fallbacks). The first revision anchored on the NEWEST occurrence, which moves the renderer's cursor forward by the duplicate gap and, when the gap is at least the page size, cycles forever — an unbounded IPC loop. The oldest-occurrence forward scan provably terminates (the next cursor is a line this page returned, strictly before the anchor, so the anchor position strictly decreases) but skips every record between two occurrences. The offset cursor makes paging exact AND terminating; the forward scan remains only as the terminating fallback.
• Why per-entry offsets rather than one oldestOffset: the renderer's cursor is the first kept line of a chunk, and chunks frequently lead with records the mapper drops (Codex turn_context/session_meta, Claude snapshots). An offset for the first returned line would fail the marker-at-offset check on every such page and silently degrade to the slow scan. Numbers only — ~2 KB per 200-record page.
• One reverse line walker (readLinesBackward) shared by both readers plus a forward twin with the same byte-level line contract, so an offset learned from one is verifiable by the other. The consumer's stop callback is what bounds the read.
• Lines across block boundaries are carried as a list of chunks and concatenated once — one copy per byte even for a multi-MB record.
• Multi-byte UTF-8 on a block boundary cannot be mis-split: 0x0A never occurs inside a multi-byte sequence, and a line is decoded only once complete.
• Head count by newlines, not by parse. Exact for provider-written transcripts (their only malformed line is the mid-append one at EOF, inside the parsed tail); a hand-edited file with junk in its head is over-counted — asserted explicitly in the test as a documented trade-off.
• A short read throws → the ordinary empty-chunk path. Both providers are append-only; a file that shrinks under us is not something to assemble lines across.
• Rejected: a growing single window re-read from EOF; estimating totalEntries from average record size (the renderer's contract is "how many records are on disk", never decremented); a backward marker hunt for the no-offset path (cannot be made to terminate on duplicates without walking the whole file anyway).

Plan: docs/superpowers/plans/2026-09-03-history-tail-read.md.

Linked issues

Fixes #747
Refs #87, #103, #93

Verification

• NODE_ENV=test npx vitest run --project unit src/main/sessions/historyLoader.test.ts — 14 passed. The oracle in the test IS the removed forward implementation (streamJsonl ring buffer / marker scan), so equivalence is against shipped behaviour: small transcript at six limits; a ~3 MB transcript with a 320 KB record spanning many blocks and a block boundary placed inside a 4-byte emoji; bytes read for the window proportional to the window while totalEntries still counts the whole file; window larger than the file; blank / malformed / literal-null / CRLF / unterminated lines; a raw U+2028 record kept whole (and pinned that readline split it); empty, junk-only and missing files; every returned offset addresses the line that parses to its entry; older pages at six anchor depths by marker AND by offset incl. the first record; Codex marker synthesis; missing-marker tail fallback; a mid-file offset reading only the page (< size/16); offsets that are another line, mid-line, past EOF, negative, fractional or absent all fall back to the forward scan with the same result; the reviewer's duplicated-marker input (0–29 = A0..A29, 30–59 = A0..A29 again, 60–79 unique, limit 10) paged to the head with the renderer's cursor rule — offset cursor: 7 pages, every record exactly once; marker-only: terminates in 4 pages, lossy across the gap exactly like the original scan.
• NODE_ENV=test npx vitest run --project unit src/main/sessions/ — 5 files, 32 passed (on the rebased branch).
• NODE_ENV=test npx vitest run --project unit src/main/remote/protocol/scope.test.ts — 22 passed (the get-history schema accepts the optional beforeOffset).
• NODE_ENV=test npx vitest run --project renderer src/renderer/src/workspace/hook/actions/ — 8 files, 35 passed.
• NODE_ENV=test npx vitest run --project system src/main/remote/RemoteServer.integration.test.ts (exercises get-history initial + older page end to end) — 13 passed.
• npx tsc -b --pretty false — clean.
• ESLint: this repository has no ESLint package or config installed, so that gate is not applicable.
• Throwaway bench (not committed) on a generated 83.9 MB / 19,556-record transcript with realistic nested tool_use / tool_result records, hot page cache, M3: initial load (limit 120) 203–217 ms → 13–16 ms, parsing 786 KB of the 84 MB; older page (limit 200) from the tail anchor 201 ms → 6 ms. Results were asserted equal to the old reader's. The real 89 MB rollout in perf(history): initial history load parses the whole transcript to return its last 120 entries #747 costs more per byte than the synthetic one (4.6 s cold / 332 ms hot).

Limitations / follow-ups

• totalEntries still requires one raw pass over the head of the file on every initial load (I/O off the main thread, a memchr-speed loop on it). Caching the count per (path, size, mtime) across loads was left out; the load runs once per session per boot.
• The no-offset path (remote client; a cursor re-anchored by the live-window trim; a failed offset check) is the pre-PR forward scan from byte 0 — terminating and lossy across duplicate markers, as before. Follow-ups that would remove it: the remote client's store adopting offsets/beforeOffset (it already receives offsets), and carrying byte offsets on live jsonl frames so a trim re-anchor keeps the exact cursor.
• The head count is by newlines; a hand-edited transcript with junk before the tail over-counts totalEntries by the junk lines (documented in the test).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kk16MNVJtWAnCeGxGRuHqa

Review repair (2026-09-04)

The renderer now selects a cursor with an explicit latch, not marker inequality. The first renderable line's byte offset advances even when its marker repeats the old marker; missing offsets clear the previous position. Added three renderer regressions, remote offset validation and real WebSocket duplicate-marker paging. 57 focused tests and TypeScript pass. Full CI passed on implementation 621bff7; the plan-only follow-up is subject to its normal CI rerun.

Final verification: both full host CI gates pass on fcebe0f (run 33938035642). Ready for owner merge confirmation; no live app restart.

[Juliusolsson05]

Review follow-up — c0f3017 (rebased onto main)

Blocker (livelock on repeated markers) — fixed with a position cursor.

• Every chunk now returns offsets (absolute byte offset of each returned record's line, parallel to entries); HistoryChunkRequest/preload/IPC/remote schema + wire types gain optional beforeOffset.
• The renderer keeps historyOldestOffset next to historyOldestMarker (state.ts), sets it from the offset of the same raw line it picks as the cursor (history.ts, initialHistory.ts), sends it back, and clears it whenever the marker comes from a live burst or a trim re-anchor (useIpcSubscriptions.ts).
• Loader: with beforeOffset it checks the byte before it is a newline and the line there carries beforeMarker, then walks back from that byte — exact, O(page). Without an offset or on a failed check it runs the original forward scan anchored on the oldest occurrence (terminating; lossy across a gap as before). The newest-occurrence backward hunt is gone, and so is the test that pinned it.
• Deviation from the suggested shape, deliberately: per-entry offsets instead of a single oldestOffset. The renderer's cursor is the first kept line, and chunks often lead with records the mapper drops (Codex turn_context/session_meta, Claude snapshots) — a first-line offset would fail the marker check on every such page and silently degrade to the slow scan.
• New tests: the minimal livelock input (0–29 = A0..A29, 30–59 = A0..A29 again, 60–79 unique, limit 10) paged to the head with the renderer's cursor rule — offset cursor reaches all 80 records exactly once in 7 pages; marker-only terminates in 4 pages, lossy exactly like the old forward scan (0–19, 50–79); an offset that is another line / mid-line / past EOF / negative / fractional / absent falls back with the forward scan's result; older pages by offset equal the oracle at six depths; every returned offset addresses the line that parses to its entry; a mid-file offset reads < size/16.

Should-fix items

• (a) Wording corrected in the loader header, the tailBytes comment, the plan and the PR body: parse cost proportional to the window; one no-decode newline count over the head (17–32 ms hot on 84 MB, a full-file read cold); the no-offset fallback parses to the anchor.
• (b) U+2028/U+2029: documented in parseJsonlLine and pinned by a test that keeps such a record whole and asserts readline split it.
• (c) PR body notes the Codex marker collisions (6,277 duplicates in 49/60 rollouts) and that the offset cursor is what makes paging exact there.
• (d) Test counts refreshed on the rebased branch.

Nits — no phantom empty line at EOF (documented), blockBytes asserted positive in both walkers, countNewlines searches only the bytes a short read filled, only returned entries are interned (internKept).

Gates — npx tsc -b --pretty false clean · src/main/sessions/historyLoader.test.ts 14 passed · src/main/sessions/ 5 files / 32 passed · src/main/remote/protocol/scope.test.ts 22 passed · renderer src/renderer/src/workspace/hook/actions/ 8 files / 35 passed · system RemoteServer.integration.test.ts 13 passed.

Left as follow-ups (in the PR body): the remote client's store adopting offsets/beforeOffset (it receives them, still pages by marker), and byte offsets on live jsonl frames so a trim re-anchor keeps the exact cursor.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kk16MNVJtWAnCeGxGRuHqa

[Juliusolsson05]

Adversarial review (re-run after the pause; PR head reviewed in the worktree).

VERDICT: APPROVE-WITH-NITS

Core engineering is correct. Traced readLinesBackward/readLinesForward byte-by-byte (phantom-EOF-line skip, pending-chunk ordering, unterminated-head carry, first-line-at-0 emission), the offset-cursor check, and both fallbacks: the exact path provably terminates (anchor strictly decreases), the marker path preserves pre-PR semantics (hasMore equivalence holds), and the limit+1 probe gives hasMore the same meaning as before. Multi-byte UTF-8 cannot mis-split (0x0A never inside a sequence). No missing-history regression: window equivalence is oracle-tested, and U+2028/U+2029 handling adds records the old reader shredded. Memory bounds hold.

1. SHOULD-FIX — No test exercises beforeOffset beyond the loader. PR body claims schema coverage, but the older-page case sends only {beforeMarker, limit} (scope.test.ts:35); beforeOffset schema acceptance, RemoteServer passthrough (RemoteServer.ts:677), and WebSocketSessionFeed.getHistory forwarding (WebSocketSessionFeed.ts:305) are all uncovered. Add one schema case and one integration case.
2. SHOULD-FIX — Renderer cursor guard stays armed when the first kept line's marker equals the current cursor marker (history.ts:126). With pages of consecutive duplicate markers (the PR's own documented 100-260-record Claude dup blocks), the cursor lands on the page's newest kept entry, so the next page advances ~1 record — O(n) pages with duplicate delivery for uuid-less entries. Not a livelock, but contradicts the "every record exactly once" invariant; its test never models this. Fix: disarm the guard after the first kept entry; add a test with a page led by a duplicate of the cursor marker.
3. NIT — finishInitialChunk derives hasMore from newline-counted total (historyLoader.ts:753,757), so junk head lines can claim hasMore where the old reader said false — one wasted request, then terminates. Documented trade-off; acceptable.
4. NIT — CRLF transcripts degrade every older page to the forward scan (historyLoader.ts:358-360 requires \n before the line). Providers write \n; worth one comment.
5. NIT — limit <= 0 now returns hasMore: false where old returned true; unreachable via real callers.
6. NIT — catch-all empty reports anchor: 'tail' even when the offset path threw mid-read (historyLoader.ts:486-496) — telemetry slightly misleading.

Commands: historyLoader 14/14 PASS; unit sessions/ 31/32 (1 pre-existing cappedTextBuffer fuzz timeout, passes at 60s — not this PR); renderer actions 35/35 PASS; RemoteServer integration 13/13 PASS; tsc -b clean.