[wasm] Report untracked GC slots for aborted Wasm R2R frames

[lewing]

Summary

Fixes a GC hole during exception dispatch in Wasm R2R code. It shows up in CI as a rare SanityCheck() assert in nested EH tests on the browser-wasm R2R leg, for example Test_throwinfinallynestedintry_30.

Root cause

Wasm R2R code has no interruptible ranges, and it reports every GC ref on the frame as an untracked (pinned) slot. A method and its funclets share those frame slots.

When an exception is thrown from a funclet, that funclet's frame is ExecutionAborted. GcInfoDecoder::EnumerateLiveSlots finds no interruptible range covering the offset and returns without reporting anything, including the untracked slots. The parent frames (the caller of an out-of-line finally, and the main method body) are skipped as already reported by the funclet.

So during dispatch nothing reports the method's untracked slots. A GC in that window can free an object still held in one of them. In the repro it's a boxed Int32. The next time the slot is reported, it points at freed memory and hits SanityCheck().

Other targets avoid this in the JIT. CodeGenInterface::setFramePointerRequiredEH forces every method with EH to be fully interruptible, because EnumGcRefs only reports slots in aborted frames that are fully interruptible. Wasm is explicitly excluded there, and Wasm R2R code can't be fully interruptible.

Fix

Add a HAS_INTERRUPTIBLE_RANGES trait to each GcInfoEncoding, false only for Wasm32GcInfoEncoding.

• EnumerateLiveSlots skips the interruptible-range handling for that encoding and reports only untracked slots outside safe points, including for aborted frames.
• The count of interruptible ranges is no longer serialized for that encoding. The encoder, the runtime decoder, the cDAC decoder and R2RDump all skip it, and DefineInterruptibleRange asserts it's never used for that encoding.
• There's no R2R version bump: Wasm hasn't shipped, so Wasm-specific format changes don't need one.

The gate is on the encoding rather than TARGET_WASM: on Wasm the interpreter's GC info goes through the same decoder, and interpreter code does define an interruptible range. Native targets are unchanged.

The cDAC GCInfoDecoder mirrors the change through the same trait (default true), and docs/design/datacontracts/GCInfo.md documents it. main has no Wasm32 GC info traits in the cDAC yet; #133890 adds them.

I fixed this in the decoder rather than having the JIT declare Wasm methods interruptible over their whole range. That claim isn't true for Wasm, and other checks rely on it (GC stress, the safe-point asserts).

Size

On browser-wasm System.Private.CoreLib.wasm (CI crossgen options), omitting the count saves 1,088 bytes (0.003%). A GC info blob only shrinks when the 2 saved bits cross a byte boundary (11,606 of 58,253 methods), and identical blobs are shared (5,331 distinct unwind entries across 60,323 methods and funclets).

Validation

I ran the CI Helix payload for Methodical_d1 from build 1613981 locally, with crossgen2 built from this branch and a browser-wasm corerun built with and without the fix. For the final format I recompiled CoreLib and the test assemblies with this branch's crossgen2.

	Without fix	With fix
Test_throwinfinallynestedintry_30, DOTNET_GCStress=0x1	same SanityCheck() assert as CI, every run	pass
throwincascadedexcept_d, throwincascadedexceptnofin_d, DOTNET_GCStress=0x1	assert, then timeout	pass
Methodical_d1 merged runner, no stress	128 passed	128 passed, same results

• Native osx-arm64 and browser-wasm checked builds pass.
• R2RDump and the cDAC unit tests pass (3179/3179).
• The final format gives the same no-stress results as a control that still writes and reads the count.
• All 84 Methodical_d1 assemblies, R2R-compiled with the checked Wasm JIT, never hit the new encoder assert.
• In every full run, the same 18 out-of-process tests fail because my local runner doesn't handle out-of-process tests.
• The final revert of the version bump only restores readytorun.h and the two ModuleHeaders files to main; it wasn't rebuilt.

With this fix, the full Methodical_d1 suite under DOTNET_GCStress=0x1 was clean in 3 of 4 runs. The 4th hit a RawGetMethodTable() assert in throw_SEH. That's a separate, pre-existing hole, #134777: CallDescrWorkerInternal leaves the call's arguments unreported while DoPrestub runs. It reproduces identically with and without this change and is fixed in #134779. With both fixes, the full suite under stress passed 4 of 4 runs (126 passed), in a run before the count was removed from the format.

I also checked native (osx-arm64, release 11.0 rc1 runtime, FullOpts JIT) with a small app. An object is held only in an address-exposed local, a finally throws, and the same method catches it with a filter that forces a GC. The object stayed alive 20/20 times, and JitDisasm shows the method as ; fully interruptible, as setFramePointerRequiredEH requires.

CI doesn't run GC stress on the Wasm R2R leg, which is why this showed up as a ~0.5% flake rather than a hard failure.

Resolves #134768

Note

This PR description was drafted with GitHub Copilot.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to 'arch-wasm': @lewing, @pavelsavara
See info in area-owners.md if you want to be subscribed.

[lewing]

cc @dotnet/wasm-contrib