Heap watchdog misses V8 old-space OOMs below global heap limit #368

Problem

The current main-process heap watchdog is calibrated against V8's global heap_size_limit, but the recent crashes abort in old-space well below that reported ceiling. As a result, the app can hit a hard V8 OOM without heap.pressure and without a heap snapshot.

Related: #327, #365, #363, #364.

Evidence

Recent retained crash runs all reported heapLimit=4294705152, but fatal old-space in the terminal logs was much lower:

• 2026-06-30T14-38-07-726Z-main-11664-2f1983
  • last heartbeat: heapUsed=1180040704, ratio 27.5%
  • fatal GC: old-space around 2733 MB
  • no heap.pressure, no heap snapshot
• 2026-07-01T10-20-42-081Z-main-52514-8cbf1a
  • last heartbeat: heapUsed=844826084, ratio 19.7%
  • perf max before crash: heapUsed=2731220256, ratio 63.6%
  • fatal GC: old-space around 2330 MB
  • no heap.pressure, no heap snapshot
• 2026-07-02T12-42-41-292Z-main-43457-dc8023
  • last heartbeat: heapUsed=2614774260, ratio 60.9%
  • fatal GC: old-space around 2731 MB
  • no heap.pressure, no heap snapshot

The local branch still has the older watchdog at roughly 75% / 3.0GiB. origin/main from #364 lowered that to 70% and adaptive 2s sampling above 50%, but 70% of this heap limit is still about 2.80GiB, above the observed fatal band.

Suspected Cause

The watchdog and journal heartbeat use global heap metrics (used_heap_size, process.memoryUsage().heapUsed, heap_size_limit). They do not record or trip on V8 per-space pressure such as old-space, large-object-space, or total_available_size.

V8 can fail a large allocation or old-space compaction below the global heap limit, so a trip ratio based only on heap_size_limit can miss exactly the failure class we care about.

A second gap: heap.pressure is recorded only after writeHeapSnapshot() succeeds. If the threshold is missed or the snapshot fails under pressure, the incident is absent.

Proposed Fix

• Add v8.getHeapSpaceStatistics() signals to heartbeat and the heap watchdog:
  • old_space.space_used_size
  • old_space.space_size
  • old_space.space_available_size
  • large_object_space.*
  • code_large_object_space.* if useful
• Add v8.getHeapStatistics().total_available_size to heartbeat/watchdog context.
• Start fast sampling much earlier, e.g. heapUsed >= 1.0-1.25GiB or 25-30% of heap limit, not 50%.
• Trip earlier, e.g. heapUsed >= min(2.2GiB, heapLimit * 0.52) or old-space used/available thresholds derived from the observed fatal band.
• Record a lightweight heap.pressure or heap.snapshot_attempt incident before writeHeapSnapshot() so failed snapshot attempts still leave evidence.
• Consider V8 near-heap-limit snapshot support as a timer-independent backstop, then correlate/relocate snapshots on next launch.

Acceptance

• A run whose main heap reaches the observed danger band around 2.0-2.3GiB records a durable heap-pressure incident before native OOM.
• Heartbeat includes enough V8 space detail to distinguish global heap slack from old-space exhaustion.
• If snapshot write fails, the incident still records the attempt and failure reason.
• Existing Crashpad minidump correlation still works for hard native aborts.