refactor(ui): stream the transcript incrementally instead of re-deriving it per token #2030

Problem

The chat transcript simulates incremental updates by re-deriving the whole view model and relying on referential equality to suppress re-renders. The increment is known at the source — every runtime delta carries its turnId — but that information is discarded when events are flattened into the messages snapshot, then guessed back downstream through memo's reference comparison.

runtime event stream   ← incremental: the delta's turnId is explicit
      ↓                ← information discarded here
messages array         ← flattened into a full snapshot
      ↓
materializeTurns()     ← every turn re-derived
      ↓
memo reference checks  ← tries to infer which turns did not change

Correctness therefore depends on every link in a long derivation chain never allocating a new object. That defence is fragile (one link breaks the whole chain), silent (nothing fails when it breaks), and non-local (you cannot tell from a call site whether allocating is safe).

chat-view.tsx:241-249 states the invariant in a nine-line comment — "every sibling gets a stable undefined and its memo skips". Nothing tests it. materialize.test.ts is 379 lines of behavioural assertions with no identity (===) assertion anywhere.

Measured impact

Real user data, largest session: 107 turns, 20283 DOM nodes, 44802px of scroll height.

One streamed answer, measured over CDP:

Per-turn attribution over a second run shows two distinct populations:

The per-second timeline confirms the split: during 6–11s of active streaming the high-frequency group wrote ~5500/s while the low-frequency group wrote zero.

How the two populations break

Both are the same architectural cause reached by different routes:

1. Per-delta. overlayShellRunUpdates re-folds from the original input on every delta and never writes its derived result back. A background command's store revision permanently leads the tool_result snapshot persisted in messages, so merged.changed stays true forever and the owning turn is rebuilt on every token. Any session with background-command history is affected, regardless of ownership or terminal status.
2. Per phase boundary. Each tool_result / text_complete triggers refreshMessages → setMessages → full materializeTurns, invalidating every turn at once. Roughly 2 × steps + tools times per answer.

The multiplier

Every re-render, however caused, is amplified into unconditional DOM writes. useLayer's context-mode ref is a fresh inline closure per render, and useLayer also returns a new layer object per render; useTooltip's composed ref depends on both, so Tooltip's layout effect tears down and re-runs on every render, unconditionally rewriting aria-describedby and the anchor-name inline style. Each footer button therefore costs one write pair per render, and the transcript carries six per turn. This is what turns "107 turns re-rendered" into 58080 DOM writes.

The defect is in @astryxdesign/core (facebook/astryx), so it ships as a dependency patch rather than a source change — but it is the same performance problem, and fixing the derivation breakages without it leaves the multiplier in place for every re-render that legitimately remains.

Ruled out

• Sidebar — removing the entire sidebar subtree from the DOM left streaming cost unchanged (62520 writes, 84/84 turns). perf(desktop): lazy-mount sidebar session trees #2021 is a separate, real problem on the idle/navigation path.
• content-visibility — forcing visible on the large turns changed nothing (33504 vs 32880 writes).
• Accessibility tree — enabling it doubled listeners (4113 → 10962) without degrading frame times.

Why this is architectural, not three bugs

Recent optimisation has moved inward one layer at a time — #1979 (idle polling), #1985 (shell subscription granularity), #2021 (sidebar mounting) — each narrowing who gets pulled into the recomputation. None questioned why a full recomputation happens at all. That approach stops at the ChatMessageSurface boundary, because inside it is the one region that genuinely must respond to tokens. #1985 explicitly concluded "the chat surface needs no memo", trusting an invariant that no test protects.

The same fragility already bit once: #472 established the "settled turn identities stay stable" invariant, and #778 broke it ten days later while adding an unrelated feature. Nothing failed, because nothing was watching.

This also runs against Astryx's own design. @astryxdesign/core ships parseMarkdownIncremental(text, state) with createIncrementalState() — it holds explicit progress and processes only what is new — and documents referential stability as an API contract ("new reference only if changed", "stable key … never changes for a segment's lifetime"), guarded by incremental.test.ts, streaming.test.ts, and parser.perf.test.ts. Astryx expects consumers to supply increments and stable identities; we hand it a rebuilt snapshot per token.

Desired outcome

The increment survives from the event stream to the view model, and referential stability becomes a tested contract rather than a comment:

• A projection layer owns turn/tool state and returns the set of turn IDs an update actually affected.
• A plain text delta affects only the tail turn. A shell-run update affects only the owning turn. A semantically identical update affects nothing.
• Unaffected turns keep object identity across both deltas and message refreshes.
• turn.tools and the timeline derive from one canonical tool map rather than two parallel structures.
• A re-render that does happen — the tail turn during streaming — costs no unconditional attribute writes.
• Identity assertions guard the invariant, so a future change cannot silently break it the way feat(runtime): add PTY and stdin control to background Bash #778 did after perf(ui): memoize the turn derivation chain and TurnView #472 established it.

Scope

Delivered as two PRs, tracked here as one problem.

PR A — Unrelated re-renders stop writing to the DOM

Patch @astryxdesign/core under patches/ to stabilise both the context-mode ref and the layer object returned by useLayer, so Tooltip's layout effect no longer tears down and re-runs on every parent render. Document the reason and removal condition per patches/README.md; report upstream.

PR B — Incremental projection replaces per-token re-derivation

A stateful projection layer owning turn/tool state and returning affectedTurnIds; turn.tools and the timeline converged onto one canonical tool authority; message refresh narrowed to the turns whose messages actually changed; identity and affected-set regression coverage.

Both start flat from the latest main and do not depend on each other: PR A is a dependency patch, PR B is a source refactor, with no overlap in the files they touch.

Acceptance criteria

• During a streamed answer with background-command history, historical turns produce zero attribute writes; only the tail turn updates.
• A parent re-render unrelated to a tooltip's own state produces no aria-describedby or anchor-name writes on its trigger.
• A shell-run update whose semantics are unchanged produces an empty affected set.
• A message refresh that adds one turn leaves all other turns referentially identical.
• Regression coverage asserts identity, not just rendered output, and fails today.
• Live-only tools, stale-live-vs-durable revision races, and source ownership display remain behaviourally unchanged (shell-run-projection.test.ts stays green).
• Re-measuring the original session over CDP shows the 7896-write population collapsing to the 216 population's order of magnitude, and that population shrinking in turn.

Alternatives considered

• Reorder the shell-run overlay in chat-view.tsx as a stopgap first. Two independent external reviews converged on this as the minimal correct fix, and it would restore usable streaming in a few lines. Skipped deliberately: the incremental projection replaces that composition wholesale, so the stopgap would be written and deleted within the same cycle.
• Memoise inside the pure projection functions (WeakMap keyed on tool/update identity). Hides state in a pure layer and makes the cache key depend on upstream reference stability.
• Virtualise the transcript. Reduces the constant factor but leaves per-token full re-derivation in place; independently worth considering for the 20k-node DOM.
• Fork or vendor the tooltip primitive instead of patching. Heavier than the defect warrants while upstream remains responsive.

Non-goals

• Sidebar mounting behaviour (perf(desktop): lazy-mount sidebar session trees #2021) — a separate problem on the idle/navigation path, confirmed non-overlapping by removing the sidebar entirely and re-measuring.
• Streaming visual design, scroll policy, or Markdown trust boundaries.
• A general performance benchmark or telemetry framework.