feat(performance): ship local evidence, history, and profiling

[Juliusolsson05]

The existing live dashboard explains current load, but it cannot answer what made an earlier interaction slow or collect a bounded diagnostic artifact. This change completes the local performance system: Agent Code continuously retains bounded metrics and incident evidence, exposes history and profiles in the Performance monitor and command palette, and saves privacy-filtered reports only to a destination the user chooses.

What changed

• Time startup, session readiness, prompt delivery, first provider/render output, IPC, transcript work, terminal writes, persistence, worktrees, orchestration, and dictation at their real boundaries.
• Detect main/renderer stalls, long tasks, memory pressure, and slow operations with sleep and visibility exclusions, hysteresis, cooldowns, per-scope captures, and bounded before/after evidence. Dropped monitoring records are reported as coverage (history points and truncated captures), not as incidents.
• Persist 1-second, 10-second, and 1-minute history tiers across restarts with peak-preserving rollups; cap timeline pages, incident counts, evidence, disk use, and in-memory queues.
• Add timeline and incident detail views plus commands that open Recordings to save a report or record a trace, with the saved path and Reveal visible.
• Add single-owner Chromium trace and main CPU profiling, picker-based heap capture, app-owned capture scratch swept at startup, owner-close cancellation, safe file reveal, and bounded shutdown flushing.
• Add deterministic benchmark/qualification tooling and an Electron helper smoke test.

Reports are local-only. There is no upload or sharing path, and reports exclude prompts, transcript text, paths, audio, environment values, stacks, credentials, and raw run IDs.

Review round 1 resolutions

Two reviewers (correctness and performance) reported 26 findings on fd14a227. All valid findings are fixed in this head:

• Incident correctness: windows are registered at BrowserWindow creation, so a renderer that never boots is detected. Captures are per scope instead of one global slot. Heartbeat staleness requires both the helper receive age and main's source-clock age, so a transport backlog is not reported as a renderer stall.
• Shutdown durability: the coordinator drains every queued batch before flushing. The worker accepts the final batch, interrupts open captures, force-records, then closes the open rollup buckets before replying.
• Clear History also resets in-memory incidents, evidence, and operation histograms, so deleted data is not re-persisted.
• History store:
  • An in-memory index replaces full rescans, and a coalesced bounded writer replaces the unbounded promise chain.
  • Tier files are compacted only once 25% has expired. Expired files and runs are deleted.
  • A torn JSONL tail is repaired before indexing or appending.
  • Status is derived from the index. Resolution is chosen by data age and coverage.
  • Capturing incidents are repaired in every retained run.
  • Report operation histograms are attributed per whole run.
• Query lane: a bounded FIFO replaces the single slot. limit is validated. The timeline keeps its last page when a read is delayed. (Round 2 replaced crash-budget refunds for slow queries with query abandonment.)
• Producer loss: a restarted producer's counter is counted as new loss, and producers flush early once a full batch is waiting. (Round 2 made this exact with per-generation baselines and spaced the flushes.)
• First-output timing: the timer is armed only by provider acceptance, so a queued prompt is cancelled instead of being timed against the running turn. This covers remote and agent-management prompts too. Renderer cancellation stays local, and the cancel IPC channel is removed.
• transcript.parse is one observation per read call instead of one per line.
• Captures: heap snapshots use a native picker with a size estimate and a disk-headroom check on both volumes. Traces, heap snapshots, and reports write their temporary files into app-owned directories instead of beside the user's destination.

Review round 2 resolutions

Two reviewers re-reviewed 43cefe77. Both requested changes, with no blockers. All valid findings are fixed:

• History safety:
  • A partial startup index can no longer lead retention to delete a week of unindexed history.
  • Each file is indexed independently.
  • A failed append re-arms tail repair.
  • A missing run folder is recreated.
  • Cross-volume moves never truncate an existing destination.
• Incident accuracy:
  • Captures are keyed by rule and scope.
  • Windows that hang before first show are detected, after a 15 s boot grace.
  • Heartbeats and main samples travel in a priority lane.
  • Silence right after dropped records is not blamed on the renderer.
  • Clock rollback is handled in the engine and the rollups.
  • The first loss count after a helper restart is a baseline.
• Helper resilience:
  • A slow query is abandoned instead of killing the helper.
  • Shutdown admits only the flush and abandons an in-flight export.
  • Incident details are rebuilt only when their summary changes.
• Loss accounting: baselines are kept per producer generation, so reloads and late reports each count exactly once.
• UI:
  • The Timeline never shows a stale page for a different range, and retries while paused.
  • Clear History reports cleared, cancelled, busy, unavailable or failed.
• First-output timing: starts at Enter for every provider and is settled once by acceptance.

Review round 3 resolutions

Both reviewers confirmed their round-2 findings as resolved on 6db43987 and reported follow-ups. All valid follow-ups are fixed:

• Stall detection: a missing heartbeat is excused only for loss that can contain heartbeats (priority-lane or failed-batch loss), and only for one staleness interval. Unrelated drops can no longer hide a frozen renderer.
• Mutating work: report exports and history clears are never abandoned at their deadline. The helper is restarted instead, so the UI never reports an outcome the helper later contradicts. In-memory evidence is reset only after a clear succeeds.
• Clear History: the UI now distinguishes "nothing was sent" from "sent but unconfirmed".
• Helper resilience: a scan that never answers restarts the helper instead of stacking stuck reads.
• Stored history: clock steps backward of up to two buckets merge into the open bucket, and larger steps start a new bucket. Only a missing runs directory counts as having no runs. A run pruned mid-write is not recreated.
• First-output timing: Codex submits are always settled in main, even when a hidden tile cancelled the local render timer.

Validation

• tsc -b (node + web projects)
• Focused suites (performance main/shared/preload, transcript loader, session recovery, Performance Monitor, command palette, workspace commands): 43 files, 371 tests passed.
• Earlier on this branch: npm run build, npm run smoke:monitoring (three consecutive packaged-helper runs), npm run benchmark:monitoring (100,000 records, 1.458 µs p99, zero drops), npm run qualify:monitoring (all 120 maximum-fleet process generations, zero process-record drops, queue and batch bytes below 2 MiB).

The cold Electron/Node helper measured 67.94–68.06 MiB RSS, below the revised 80 MiB helper target documented in the qualification record. Eight-hour active use, 24-hour disk growth, signed arm64, and Intel measurements remain rollout qualification evidence.

Closes #956
Part of #944

🤖 Generated with Claude Code

https://claude.ai/code/session_01WKUB1nGkTZW1BhtNSXuR6M