Repository navigation
Heap: main-process OOM under heavy multi-agent orchestration — follow-up to #288 #327
Copy link
Copy link
Closed
Labels
class:C6-unboundedUnbounded growthUnbounded growthneeds-evidenceNeeds a recording/real use to settleNeeds a recording/real use to settlesev:P0Data loss, crash, app unusableData loss, crash, app unusabletype:perfPerformance or resource usePerformance or resource use
Description
Symptom
The Electron main process crashed with a hard V8 OOM during a long working session:
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
This is a follow-up to #288, not a regression of it. #288 dropped the steady-state main-process heap drastically (derive-and-drop in SubAgentWatcher, string interning, etc.), but a hard OOM crash means something is still growing unbounded — a stable-but-high heap can't CALL_AND_RETRY_LAST.
Two hypotheses
- A secondary accumulator Main-process heap unbounded — full transcript + tool-output text retained in memory (681MB, 92% strings) #288 didn't cap. Main-process heap unbounded — full transcript + tool-output text retained in memory (681MB, 92% strings) #288 targeted
SubAgentWatcher; a sibling structure (orchestration run-state, proxy flow state, history) may have the same unbounded-growth shape. - Heavy multi-agent orchestration specifically. The crash happened during a marathon orchestration session (many
orchestration_create_agentchildren + reviewers). Each child's run-state, transcripts, and the orchestration MCP's retained run outputs all live in the main process, and several very large tool results were materialized there. The orchestration MCP (agent_code) disconnected/reconnected mid-session, consistent with it being the hog this time.
Diagnostic plan
- Restart the app; reproduce by running a heavy orchestration session (several children + reviewers) and watch RSS climb.
- Capture a main-process heap snapshot at high RSS; run
scripts/analyze-heapsnapshot.mjs --owners <snap>to name the dominator (orchestration run-state vs. a new accumulator vs.SubAgentWatcheragain). - Check the unbounded disk-hog dirs that can also bleed into memory:
~/.config/agent-code/{feed-debug,debug-bundles,proxy,performance}. - If orchestration is the cause: bound/evict retained child run outputs + transcripts in the main process (derive-and-drop, mirroring the Main-process heap unbounded — full transcript + tool-output text retained in memory (681MB, 92% strings) #288 fix).
Context
Surfaced while building the conditions-framework stack (integration/conditions-framework, PRs #325/#326) under heavy orchestrated review. Related: #288 (closed), #103 (perf sweep).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
class:C6-unboundedUnbounded growthUnbounded growthneeds-evidenceNeeds a recording/real use to settleNeeds a recording/real use to settlesev:P0Data loss, crash, app unusableData loss, crash, app unusabletype:perfPerformance or resource usePerformance or resource use
Activity
Juliusolsson05 commentedon Jul 9, 2026
Two structural mitigations for this OOM class merged today: PR #510 byte-caps the orchestration output pipeline end to end (a single huge child answer can no longer be stringified/mirrored unbounded into MCP responses, bridge caches, and 24h tombstones — worst-case main-process tombstone retention drops from 500×100×unbounded to a hard 256K/agent), and PR #511 bounds live renderer transcript state (count+byte window with safe pagination reload) and adds the per-session memory instrumentation needed to attribute the next incident. Root-cause investigation for the orchestration-heavy reproduction continues in #369.
Juliusolsson05 commentedon Sep 27, 2026
Closing as a duplicate of #365, which is the evidence-driven follow-up (triage 2026-09-27, main
a70403ef).This keeps a single memory/crash issue that works from evidence, not two overlapping ones.
Reopen when: a heap-pressure incident or heap snapshot shows memory growing again. Attach the incident id and the snapshot.