JIT: keep the block a catch returns to non-empty when optimizations are disabled

[pcshrosbree]

Fixes #135637.

When a catch block ends with a thread abort pending, the runtime raises the abort again from the address the catch returns to and looks the handlers up for that address (EECodeManager::ResumeAfterCatch → COMPlusCheckForAbort → ThrowControlForThread, which dispatches from a FaultingExceptionFrame at the exact resume address). The JIT already accounts for that: impImportLeave places the block a catch returns to inside the EH regions that enclose the catch (see its comment on ThreadAbortException), and fgOptimizeEmptyBlock keeps that block from being empty when it is in a different EH region than its successor, by adding a GT_NO_OP, "for re-raise of thread abort exceptions to work". The second part only runs when optimizing: it is part of fgUpdateFlowGraph, which asserts OptimizationEnabled().

So in MinOpts and debuggable code that block falls into its successor without generating code, the resume address is the end of the block's EH regions, and the abort is dispatched as if it had been raised in the code that follows. In the shapes of #135637 a finally the catch's leave passes through does not run, an enclosing catch does not see the abort, and a try that starts right after the catch does.

This change moves the check into a helper, fgKeepCatchRetTargetNonEmpty, and runs it from fgPostImportationCleanup when optimizations are disabled. fgOptimizeEmptyBlock calls the same helper, so optimized code is unchanged. The change inserts a GT_NO_OP into each empty block a catch returns to that is in a different EH region than its successor, when optimizations are disabled; that is one nop instruction per such block.

What changes

Checked builds on linux-x64, the JIT from this branch against a JIT built at main 523a36f on the same runtime; three runs per configuration and shape, identical each time. The traces are the handlers that ran; "finally ran" is a call-free volatile store.

shape	tier 0 / JITMinOpts=1 before	tier 0 / JITMinOpts=1 after	TieredCompilation=0 (before = after)
CatchThenFinally	catch outer, finally not run	catch finally outer, finally ran	catch finally outer, finally ran
LeaveOutOfOuterTry	inner	inner outer	inner outer
ConsecutiveTry	inner new catch	inner	inner

In the tier-0 JitDump of CatchThenFinally the catch-return block BB03 is now a one-byte nop at 0x1B and the finally's try range covers it (EH#1: try [0014..001C); before: [0014..001B) and no code for BB03); the no-op is added in the Post-import phase. The TieredCompilation=0 layouts of the three methods (block list, instruction group offsets, EH table, code size) are identical before and after.

Tests

Regressions/coreclr/GitHub_135637/test135637 has the three shapes. With the current JIT it fails under tier 0 and JITMinOpts=1 and passes under TieredCompilation=0; with this change it passes in all three. That pattern reproduces on linux-x64, on osx-arm64, and on windows-x64 with #135531 applied: on a machine with CET shadow stacks the CatchThenFinally shape hits the Checked-only assert of #135530 in CallCatchFunclet with and without this change, because the re-raised abort is caught in the same method, so this test depends on #135531 and should merge after it.

The test runs on jitted code only: in the recorded interpreter runs (DOTNET_InterpMode=1 and 3) the finally runs, but the outer catch of LeaveOutOfOuterTry does not; I will raise that with the interpreter owners separately.

JIT/Methodical (Methodical_d2, _r2, _do, _ro, _others), JIT/jit64 (jit64_5) and baseservices/exceptions on linux-x64 under the default configuration, TieredCompilation=0 and JITMinOpts=1, with the JIT before and after the change: no result differs. 74 of the 79 runner scripts pass with both JITs, and the same five (baseservices/exceptions/inproccrashreport/*) fail with both on that machine.

Tier-0 code size, jitting with ReadyToRun=0 (CoreLib and the repro program, three runs per JIT, linux-x64): of 2467 tier-0 methods whose size is stable across the runs, 4 grow by one byte each (the three shapes of the repro and System.Diagnostics.StackFrameHelper:InitializeSourceInfo); the compared FullOpts and Tier1 methods are unchanged. On arm64 the nop is four bytes.

Note

This PR was prepared with AI assistance (Anthropic Claude, checked by OpenAI Codex) at my request. I reviewed the change, the test and the measurements before opening it.

[azure-pipelines]

Azure Pipelines:
Successfully started running 5 pipeline(s).
11 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[JulieLeeMSFT]

@EgorBo PTAL.