Repository navigation
Runtime: prove generator and iterator side-store heap-cap recovery under no-GIL #36
Description
Activity
Landed the first #36 policy witness in
zig-js@9a10bb7c(test(oom): cover deferred generator recovery guard).What changed:
- Added a focused no-GIL heap-cap recovery test that roots a suspended generator, then directly attempts allocation-failure recovery while a peer is live.
- The test asserts the abort-safe parallel collector does not report recovery/sweep when generator tracing is deferred to a world-stopped finish.
- Telemetry now pins the intended boundary: attempts increase, collections do not, round-limit aborts increase, and
gc_par_deferred_roundsis observed. - Updated
docs/threads/limits.mdanddocs/threads/testing.mdto distinguish trace-sensitive side-store locks from deferred generator/iterator tracing.
Validation:
/Users/chris/.local/bin/zig build test -Dtest-filter="deferred generator tracing" --summary all/Users/chris/.bun/bin/bun run docs:buildgit diff --check
CI for this push is pending: https://github.com/zig-utils/zig-js/actions/runs/29200654888
This does not implement a positive recoverable generator/iterator class yet; it makes the fail-closed invariant executable before we try to broaden recovery.
Follow-up #36 policy witness landed in
zig-js@97c511c3(test(oom): cover iterator helper recovery guard).What changed:
- Added the iterator-helper sibling to the deferred-generator recovery guard.
- The new test roots a mid-helper
Iterator.from(...).map(...).filter(...)state, then attempts no-GIL allocation-failure recovery while a peer is live. - It asserts the parallel collector still fails closed: attempts increase, collections do not, abort/round-limit telemetry increases, and
gc_par_deferred_roundsis observed. - Updated the testing guide to say both deferred generator and deferred iterator-helper witnesses cover this boundary.
Validation:
/Users/chris/.local/bin/zig build test -Dtest-filter="deferred iterator helper tracing" --summary all/Users/chris/.bun/bin/bun run docs:buildgit diff --check
CI for this push is pending: https://github.com/zig-utils/zig-js/actions/runs/29200732250
The negative/fail-closed half of #36 now covers both deferred cell kinds called out by the parallel tracer. The remaining #36 work is to decide whether any generator/iterator state has a positive recoverable class under live no-GIL peers.
Follow-up #36 request-buffer witness landed in
zig-js@5b5f310d(test(oom): cover async generator recovery guard).What changed:
- Added a focused no-GIL heap-cap recovery guard for async-generator request side stores.
- The test creates an async generator suspended on an unresolved
await, queues later.next()/.return()requests, and asserts the request backing store is live before recovery is attempted. - Allocation-failure recovery is then attempted while a peer is live; the expected fail-closed telemetry is pinned: attempt increases, collection count does not, round-limit aborts increase, and
gc_par_deferred_roundsis observed. - Updated the testing guide so Runtime: prove generator and iterator side-store heap-cap recovery under no-GIL #36's fail-closed coverage lists suspended generators, pending async-generator requests, and iterator helpers.
Validation:
/Users/chris/.local/bin/zig build test -Dtest-filter="deferred async-generator requests" --summary all/Users/chris/.bun/bin/bun run docs:buildgit diff --check
CI for this push is pending: https://github.com/zig-utils/zig-js/actions/runs/29200819725
The negative/fail-closed witnesses now cover the deferred generator cell family called out by #36: suspended generator stack, pending async-generator requests, and iterator-helper state. Remaining work is the design/proof for any positive recoverable state under live no-GIL peers.
Follow-up #36 handler-buffer witness landed in
zig-js@8c137d53(test(oom): cover generator handler recovery guard).What changed:
- Added a focused no-GIL heap-cap recovery guard for generator handler storage.
- The test suspends a generator inside
try/finally, proving GC-backed resumable handler/execution buffers are live before recovery is attempted. - Allocation-failure recovery is attempted while a peer is live; it must fail closed with attempts/round-limit/deferred telemetry increasing and no collection reported.
- Updated the testing guide so Runtime: prove generator and iterator side-store heap-cap recovery under no-GIL #36's fail-closed coverage now explicitly lists suspended stack, resumable handler, pending async-generator requests, and iterator-helper state.
Validation:
/Users/chris/.local/bin/zig build test -Dtest-filter="deferred generator handlers" --summary all/Users/chris/.bun/bin/bun run docs:buildgit diff --check
CI for this push is pending: https://github.com/zig-utils/zig-js/actions/runs/29200885804
This fills the generator handler-storage part of the #36 reproduction checklist while preserving the current fail-closed policy.
Telemetry follow-up landed in
4b20837b: deferred generator/iterator blocked finishes now incrementgc_par_deferred_aborts, the #36 fail-closed witnesses assert it, andmidgc-profileprints it asdef-abort. Current CI: https://github.com/zig-utils/zig-js/actions/runs/29201026809Routing note: the previous #36 head CI run for
4b20837bwas cancelled by the newer #15 profiler push. The #36 code remains onmain; use the combined-head CI run for03092fb0as the current full gate: https://github.com/zig-utils/zig-js/actions/runs/29201263212 (queued at the time of this note).Routing note: the combined-head CI run for
03092fb0was superseded by the newer #13 workflow-support push. The #36 fail-closed generator/iterator recovery witnesses and deferred-abort telemetry remain onmain; use current combined-head CI for36c0159a: https://github.com/zig-utils/zig-js/actions/runs/29201371564 (queued at the time of this note).Routing note: the combined-head CI run for
36c0159awas cancelled by the newer #16 nursery-profile push. The #36 fail-closed recovery witnesses and deferred-abort telemetry remain onmain; use current combined-head CI forb79f9774: https://github.com/zig-utils/zig-js/actions/runs/29201561685 (queued at the time of this note).Routing note: current combined-head CI moved with the #12 memory-model docs push. The #36 recovery witnesses and deferred-abort telemetry remain on
main; use current combined-head CI ford0092f5e: https://github.com/zig-utils/zig-js/actions/runs/29201642565 (queued at the time of this note).Closed as the deliberate all-fail-closed policy after the combined
maingate ford0092f5epassed: https://github.com/zig-utils/zig-js/actions/runs/29201642565Evidence now covered by the green gate:
- suspended generator execution storage fails closed under live no-GIL peers;
- resumable generator handler buffers fail closed;
- pending async-generator request buffers fail closed;
- iterator-helper backing state fails closed;
- deferred-cell abort telemetry (
gc_par_deferred_aborts/def-abort) remains separate from round-limit abort accounting; - docs distinguish trace-sensitive locks from deferred generator/iterator tracing.
No positive recoverable generator/iterator class is sound today. If one appears later, it should be reopened/narrowed with proof that no deferred edges remain or that recovery can complete through a sound world-stopped path.
Parent: #30
Goal
Define and implement the safe recovery policy for GC-backed generator / iterator-helper side-store allocation pressure under live no-GIL peers.
Why this is separate
#30 has proven no-GIL emergency recovery for GC-cell slabs and safepoint-owned
ArrayBufferbyte slabs, while trace-sensitive Object / Promise / Environment / async-generator request critical sections intentionally fail closed.Generator and iterator-helper side stores are a different seam: the parallel tracer defers their mutable storage to the world-stopped finish path because a running generator's
execstack, async-generator request state, resumable handler buffers, and iterator-helper backing state are not sound to trace directly while mutators run. The allocation-failure collector refuses to sweep while those deferred edges remain, so treating these as an ordinary side-store retry class would either fail to reclaim or risk weakening the tracing invariant.Current status
The negative/fail-closed policy is now executable for the deferred generator/iterator family:
gc_par_deferred_aborts/def-abort, while still preserving the existing round-limit abort accounting identity.docs/threads/limits.mdanddocs/threads/testing.mddistinguish trace-sensitive locks from deferred generator/iterator tracing.The current design evidence points to a conservative rule: deferred generator/iterator state should not be recovered by the live no-GIL allocation-failure collector unless a future allocation site can prove no deferred edges remain, or the collector can finish through a sound world-stopped path. No positive recoverable generator/iterator class has been proven yet.
Closure policy
Decision after the combined
maingate ford0092f5epassed (https://github.com/zig-utils/zig-js/actions/runs/29201642565): no current generator/iterator side-store allocation class is sound to recover under live no-GIL peers. The implemented policy is intentionally all-fail-closed while deferred generator/iterator edges remain pending.This preserves the tracer invariant: a parallel allocation-failure collector may not claim a sweep while suspended generator stacks, resumable handler buffers, async-generator request buffers, or iterator-helper backing state have been deferred to the world-stopped finish path.
Future work should reopen this tracker or file a narrower follow-up only if a specific allocation site can prove either that no deferred edges remain or that recovery can finish through a sound world-stopped path. Any such positive class needs a focused witness plus the existing negative fail-closed coverage.
Acceptance criteria