perf(diagnostics): move the session recorder and feed-debug sinks off the main process #1346

Activity

1. Juliusolsson05

   added

   type:perfPerformance or resource usePerformance or resource use

   class:C6-unboundedUnbounded growthUnbounded growth

   sev:P3MinorMinor

   on Sep 27, 2026
2. Juliusolsson05 commented on Sep 27, 2026

   

   Juliusolsson05

   on Sep 27, 2026

   OwnerAuthor

   More actions

   Measurement before building (W3, 2026-09-27): moving these sinks to a worker would make main slower, not faster.

   The test used one real feed-debug file from this machine: 16,832 records, 134 MB, Node 24.14.1, 3 runs each. The time is main-thread CPU to hand every record off:

   Hand-off	Time
   JSON.stringify each record (what main does today)	333 ms
   v8.serialize each record (the structured-clone wire format postMessage and utilityProcess use)	5,029 ms
   structuredClone each record	13,955 ms

   What follows from this:

   • Posting objects to a utilityProcess or worker structured-clones them on main, which costs about 15× the stringify we would be moving.
   • Posting a pre-stringified string still leaves the stringify on main (the part the issue wanted moved). The only thing left to move is the file write, and fs/promises already does that on libuv's thread pool, off the main JS thread.
   • So the proposed architecture removes little or nothing from the main loop and adds a process, a protocol and backpressure.

   This file is feed-debug. The session recorder's screen frames are strings plus small objects, the same shape, so I expect the same result, but no session recording exists locally to measure it.

   Proposal: close this as not worth it, or re-scope it to what does help on main:

   • time-slice queueFeedDebugAppend's stringify the way the recorder already does (≤ 8 ms chunks with setImmediate yields);
   • keep the perf(diagnostics): default-off and off-thread the always-on diagnostic subsystems #767 opt-in gate so this cost exists only while diagnostics are on.

   That re-scope overlaps open #1392 (feedDebugLog.ts), so it would follow #1392. I am waiting on the manager's call before building anything.

   🤖 Generated with Claude Code