perf(sessions): run session index and history loads off the main thread #769

Activity

1. Juliusolsson05

   added

   type:perfPerformance or resource usePerformance or resource use

   class:C6-unboundedUnbounded growthUnbounded growth

   sev:P2Real bug with a workaroundReal bug with a workaround

   on Sep 25, 2026
2. Juliusolsson05 commented on Sep 27, 2026

   

   Juliusolsson05

   on Sep 27, 2026

   OwnerAuthor

   More actions

   Re-check on origin/main b3a2c481 before building anything (W2, quality loop).

   What changed since this was filed: sessionIndex no longer exists; the conversations catalog replaced it. Transcript reads and parses still all run on the main process thread, though. Nothing uses worker_threads, and the three utilityProcess children don't touch transcripts. The heavy paths today:

   • historyLoader initial tail: 256 KiB blocks parsed synchronously, plus countNewlines over the whole file. The older-chunk marker fallback parses from byte 0.
   • promptFolder.extractPromptsFromFile: its window grows ×4 up to 16 MiB, and each window is one synchronous split plus parse.
   • Catalog search: up to 150 rows, and a cold row reads at least 256 KiB before the budget check.
   • The first claudeHistory refresh parses the whole ~/.claude/history.jsonl.
   • transcriptEngine.readStableTranscript reads the whole file for rewind, duplicate and switch.

   What the owner's recordings say (~/.config/agent-code/performance-monitor/runs, all runs):

   • transcript.read: 2,436 samples, mean 178 ms. Only 3 slow-operation incidents (1.0 s, 1.3 s, 1.8 s).
   • 14 main-stall incidents, none within 10 s of a slow transcript.read. Main-stall incidents carry no operation attribution, so the recordings can't say what does stall main.
   • Two measurement defects:
     • Each IPC initial load emits two transcript.read samples. One span ends at path resolution (result: 'delegated') before the read, which skews the percentiles low.
     • sessionIndex.extractPrompts, the catalog's prompt extraction and likely the largest synchronous parse, is not mapped into the monitor at all. Catalog search and indexing have no monitor operation.

   Proposed order:

   1. Measure first: fix the double sample, and map catalog extraction, search and index refresh into monitor operations.
   2. Then move whichever path the data shows stalling main into a worker, with a build entry like performanceWorker and a verify-build-output update.

   Moving historyLoader alone, as the issue asks, is not supported by the current recordings.

   🤖 Generated with Claude Code
3. Juliusolsson05

   added a commit that references this issue on Sep 27, 2026

   fix(performance): size histograms for every operation, close failed d…

   ...

   5610f2d