Skip to content

Heap: main-process OOM under heavy multi-agent orchestration — follow-up to #288 #327

Closed

Description

@Juliusolsson05

Symptom

The Electron main process crashed with a hard V8 OOM during a long working session:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

This is a follow-up to #288, not a regression of it. #288 dropped the steady-state main-process heap drastically (derive-and-drop in SubAgentWatcher, string interning, etc.), but a hard OOM crash means something is still growing unbounded — a stable-but-high heap can't CALL_AND_RETRY_LAST.

Two hypotheses

  1. A secondary accumulator Main-process heap unbounded — full transcript + tool-output text retained in memory (681MB, 92% strings) #288 didn't cap. Main-process heap unbounded — full transcript + tool-output text retained in memory (681MB, 92% strings) #288 targeted SubAgentWatcher; a sibling structure (orchestration run-state, proxy flow state, history) may have the same unbounded-growth shape.
  2. Heavy multi-agent orchestration specifically. The crash happened during a marathon orchestration session (many orchestration_create_agent children + reviewers). Each child's run-state, transcripts, and the orchestration MCP's retained run outputs all live in the main process, and several very large tool results were materialized there. The orchestration MCP (agent_code) disconnected/reconnected mid-session, consistent with it being the hog this time.

Diagnostic plan

  • Restart the app; reproduce by running a heavy orchestration session (several children + reviewers) and watch RSS climb.
  • Capture a main-process heap snapshot at high RSS; run scripts/analyze-heapsnapshot.mjs --owners <snap> to name the dominator (orchestration run-state vs. a new accumulator vs. SubAgentWatcher again).
  • Check the unbounded disk-hog dirs that can also bleed into memory: ~/.config/agent-code/{feed-debug,debug-bundles,proxy,performance}.
  • If orchestration is the cause: bound/evict retained child run outputs + transcripts in the main process (derive-and-drop, mirroring the Main-process heap unbounded — full transcript + tool-output text retained in memory (681MB, 92% strings) #288 fix).

Context

Surfaced while building the conditions-framework stack (integration/conditions-framework, PRs #325/#326) under heavy orchestrated review. Related: #288 (closed), #103 (perf sweep).

Activity

  1. Juliusolsson05 commented on Jul 9, 2026

    @Juliusolsson05
    OwnerAuthor

    Two structural mitigations for this OOM class merged today: PR #510 byte-caps the orchestration output pipeline end to end (a single huge child answer can no longer be stringified/mirrored unbounded into MCP responses, bridge caches, and 24h tombstones — worst-case main-process tombstone retention drops from 500×100×unbounded to a hard 256K/agent), and PR #511 bounds live renderer transcript state (count+byte window with safe pagination reload) and adds the per-session memory instrumentation needed to attribute the next incident. Root-cause investigation for the orchestration-heavy reproduction continues in #369.

  2. Juliusolsson05 commented on Sep 27, 2026

    @Juliusolsson05
    OwnerAuthor

    Closing as a duplicate of #365, which is the evidence-driven follow-up (triage 2026-09-27, main a70403ef).

    This keeps a single memory/crash issue that works from evidence, not two overlapping ones.

    Reopen when: a heap-pressure incident or heap snapshot shows memory growing again. Attach the incident id and the snapshot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    class:C6-unboundedUnbounded growthneeds-evidenceNeeds a recording/real use to settlesev:P0Data loss, crash, app unusabletype:perfPerformance or resource use

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions