runtime: deep generator / async-generator / Promise execution semantics (post-#4374 remainder)

Summary

After the recent generator descriptor/funcid/return-throw work
(#4366, #4374, #4383), built-ins/GeneratorPrototype is down to ~15 failures and
built-ins/AsyncGeneratorPrototype ~36 (verified 2026-06-04 on main, uncapped).
The remainder are deep execution-semantics gaps, grouped below. Filing as one
tracker with discrete, independently-shippable sub-items.

Sub-items

B1 — `[[GeneratorState]] == "executing"` re-entrancy guard (clean, ~6 tests)

Resuming a generator that is currently running must throw a TypeError and mark it
completed. Perry doesn't track the "executing" state, so the re-entrant call
silently no-ops.

var iter;
function* g() { iter.next(); }   // re-entrant resume
iter = g();
// Node: iter.next() throws TypeError; Perry: no throw

Tests: GeneratorPrototype/{next,return,throw}/from-state-executing.js
(+ the AsyncGeneratorPrototype mirrors). Needs an executing flag on the
generator state machine checked at the top of next/return/throw resume.

B2 — `.throw()` / `.return()` try-finally state-machine edge cases (deep)

#4374/#4383 landed the core (run pending finally on abrupt completion;
.throw continues after catch). Remaining failures are nested/ordering cases:
GeneratorPrototype/{return,throw}/try-finally-nested-try-catch-within-*.js,
throw/try-catch-{within,following}-*.js. These are continuation-cloning edge
cases in linearize.rs — extends the same machinery as #4374.

B3 — AsyncGeneratorPrototype request-queue & promise ordering (deep)

AsyncGeneratorPrototype/{next,return,throw}/request-queue-*.js,
return-suspended*-promise*.js, throw-suspended*-promise*.js — the async
generator request queue and its interaction with promise resolution/await
ordering. Related to #4405 (for-await over a variable-held async generator).

B4 — Promise microtask / `then` ordering (adjacent, less-probed)

built-ins/Promise/* shows ~20 runtime + ~6 diff failures, several around
microtask/then callback ordering and resolution sequencing. Needs its own
focused probe to split into discrete causes before fixing — listing here so the
async-semantics work is tracked in one place; split out once probed.

Notes

Repro env: PERRY_NO_AUTO_OPTIMIZE=1 PERRY_NO_CACHE=1 PERRY_ALLOW_UNIMPLEMENTED=1
vs node --experimental-strip-types; assemble test262 cases via
scripts/test262_subset.assemble (onlyStrict tests need the "use strict"
prefix the radar adds — a bare .ts runs sloppy/CommonJS under strip-types and
gives misleading results).
B1 is the clean starter; B2/B3 are deep and overlap with in-flight generator
work (generator: .return()/.throw() must run pending try/finally (and .return must honor catch) on abrupt completion #4374, for await over a variable holding an async generator yields undefined (falls to array desugar) #4405) — coordinate to avoid collision.

Part of the parity roadmap #4315.

I’ll take the B1 slice here: the [[GeneratorState]] == "executing" re-entrancy guard for generator resume methods, with focused reproduction before the fix. I’ll leave the deeper try/finally and async-generator queue sub-items for separate follow-up.

Opened draft PR #4445 for the sync-generator B1 slice: #4445

Scope:

Adds an __gen_executing guard to lowered sync generator next / return / throw closures.
Covers both uncaught re-entry, which completes the generator, and caught re-entry, which lets the active body continue.
Adds a focused parity fixture for those sync cases.
Includes a small current-main compile repair in the build-cache stats plumbing, needed before this branch could build.

Validation:

Shared-target debug build for perry and perry-runtime.
Focused Node 26 vs Perry fixture diff is empty: 15 stdout lines each, 0 stderr lines each.
cargo fmt --all -- --check
jq empty test-parity/known_failures.json test-parity/threshold.json test-parity/parity_matrix_baseline.json
./scripts/check_file_size.sh
git diff --check
cargo test -q -p perry-transform

I left the async-generator queue behavior out of this PR; probing it overlaps with the B3 request-queue semantics in this tracker.

@andrewtdiz

B2 progress — PR #4457 (try/catch in the generator state machine)

Took the B2 slice. Probing turned up a root cause deeper than the issue's
framing ("continuation-cloning edge cases in linearize.rs"): for sync
generators the catch handler was entirely absent from the normal .next()
dispatch — it lived only inside the .throw() closure. That single gap
produced two distinct symptom clusters:

a throw executing inside a try during a plain .next() was never
caught — it propagated out of next() (the [object] escaped failures);
a yield inside a catch was swallowed — rewritten to await in the throw
closure (catch (e) { yield e }).

PR #4457 makes try/catch real in the state machine: the catch body is linearized
into its own states, the dispatch loop is wrapped in a try/catch that routes
thrown errors to the matching catch_entry_state, and .throw() routes the same
way. Async generators are untouched (the linearized catch states are dead on the
async path → byte-identical).

Baseline (test262, PERRY_NO_AUTO_OPTIMIZE=1 PERRY_NO_CACHE=1, vs node --experimental-strip-types) on GeneratorPrototype/{throw,return}/try-*:
12 failing → 6 failing (15 → 21 pass). Broader sweep of every try-containing
generator test: 0 regressions, 17 newly-fixed — the 6 here plus 11
language/expressions/yield yield*-delegation error-handling cases
(star-rhs-iter-*-err, from-catch, rhs-unresolvable).

Remaining B2 (separate follow-up)

The 6 still-failing cases all involve a yielding finally (finally { yield N }):

throw/try-finally-within-try
throw/try-finally-nested-try-catch-within-{catch,finally,outer-try-after-nested,outer-try-before-nested}
return/try-finally-nested-try-catch-within-finally

These need a completion-record mechanism distinct from the try/catch work:
save the pending abrupt completion (throw value / return value), route control
into the finally's yielding states, and re-raise the saved completion once the
finally finishes. Worth its own sub-item (call it B2-finally) — it does not
overlap the try/catch machinery PR #4457 lands.

B1 ([[GeneratorState]] == "executing") is still @andrewtdiz's; B3/B4 untouched.

fix(generator): make try/catch real in the sync generator state machi…

fix(generator): route abrupt completion through yielding finally (#4438…

B2 try-finally complete — PR #4474 (yielding-finally completion records)

Follow-up to #4457 (merged). The 6 cases that remained all involved a yielding
finally (finally { yield N }): abrupt completion while suspended in the
protected try never ran the finally.

PR #4474 implements completion records — a yielding finally gets a clean entry
state + a completion-check state; abrupt completion routes to the innermost
enclosing handler (catch or yielding finally) via one ordered routing chain
shared by the dispatch wrapper, .throw(), and .return(); the finally's
completion check re-raises the pending throw/return. Rebased cleanly over #4445
(B1 executing-state guard) — both coexist.

GeneratorPrototype/{throw,return}/try-*: 21 → 27 pass — 0 failing.
Regression (vs current main incl. B1): sync try-generators 0 regressions /
6 fixed; async 0 regressions (byte-identical); B1's from-state-executing
tests still pass; repo gap tests + perry-transform unit tests green.

This closes the B2 try-finally work. Remaining on this tracker: B3
(AsyncGeneratorPrototype request-queue / promise ordering) and B4 (Promise
microtask/then ordering). B1 landed in #4445.

fix(generator): route abrupt completion through yielding finally (#4438…

I am taking a focused B3 async-generator request-queue slice here.\n\nPlanned scope: reproduce one or two current-main request-queue / promise-ordering mismatches, add a focused parity fixture, and fix the local async-generator queue/resume path if the root cause is bounded. I will avoid the already-merged sync-generator B1/B2 work, broad Promise scheduler rewrites, and unrelated lowering changes unless the reproduction proves they are part of the same path.

Clarifying my B3 claim above after a shell quoting issue stripped two terms from the text.

Planned scope: reproduce one or two current-main AsyncGeneratorPrototype request-queue / promise-ordering mismatches, add a focused parity fixture, and fix the local async-generator queue/resume path if the root cause is bounded. I will avoid the already-merged sync-generator B1/B2 work, broad Promise scheduler rewrites, and unrelated for await lowering changes unless the reproduction proves they are part of the same path.

Published a draft PR for this slice: #4492

Validation completed:

cargo check -q -p perry-runtime --lib
cargo build --release -p perry-runtime -p perry with shared CARGO_TARGET_DIR and CARGO_BUILD_JOBS=1
cargo fmt --all -- --check
git diff --check and git diff --cached --check
./scripts/check_file_size.sh
cargo test -q -p perry-runtime test_gc_init_mutable_scanner_families_rewrite_runtime_slots
focused direct Node 26 vs Perry parity diffs for the new async-generator queue fixture plus adjacent generator, stream, and Promise fixtures

fix(transform): try/catch inside an async generator never caught anyt…

fix(async): linearize `await` inside `catch` for async fns/closures (P…

Navigation Menu

Repository navigation

runtime: deep generator / async-generator / Promise execution semantics (post-#4374 remainder) #4438

Summary

Sub-items

B1 — `[[GeneratorState]] == "executing"` re-entrancy guard (clean, ~6 tests)

B2 — `.throw()` / `.return()` try-finally state-machine edge cases (deep)

B3 — AsyncGeneratorPrototype request-queue & promise ordering (deep)

B4 — Promise microtask / `then` ordering (adjacent, less-probed)

Notes

B2 progress — PR #4457 (try/catch in the generator state machine)

Remaining B2 (separate follow-up)

B2 try-finally complete — PR #4474 (yielding-finally completion records)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

runtime: deep generator / async-generator / Promise execution semantics (post-#4374 remainder) #4438

Description

Summary

Sub-items

B1 — [[GeneratorState]] == "executing" re-entrancy guard (clean, ~6 tests)

B2 — .throw() / .return() try-finally state-machine edge cases (deep)

B3 — AsyncGeneratorPrototype request-queue & promise ordering (deep)

B4 — Promise microtask / then ordering (adjacent, less-probed)

Notes

Activity

andrewtdiz commented on Jun 4, 2026

andrewtdiz commented on Jun 4, 2026

proggeramlug commented on Jun 4, 2026

B2 progress — PR #4457 (try/catch in the generator state machine)

Remaining B2 (separate follow-up)

proggeramlug commented on Jun 4, 2026

B2 try-finally complete — PR #4474 (yielding-finally completion records)

andrewtdiz commented on Jun 5, 2026

andrewtdiz commented on Jun 5, 2026

andrewtdiz commented on Jun 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

B1 — `[[GeneratorState]] == "executing"` re-entrancy guard (clean, ~6 tests)

B2 — `.throw()` / `.return()` try-finally state-machine edge cases (deep)

B4 — Promise microtask / `then` ordering (adjacent, less-probed)