Assertion 'codeManager->IsUnwindable(pvAddress) || runtime->IsConservativeStackReportingEnabled()' in System.Runtime.Tests on ARM64 #134912

Activity

1. MichalStrehovsky

   added

   arch-arm64

   area-NativeAOT-coreclr

   on Sep 30, 2026
2. github-project-automation

   added this to AppModelon Sep 30, 2026
3. dotnet-policy-service

   added

   untriagedNew issue has not been triaged by the area ownerNew issue has not been triaged by the area owner

   on Sep 30, 2026
4. dotnet-policy-service commented on Sep 30, 2026

   

   dotnet-policy-serviceBot

   on Sep 30, 2026 – with Dotnet Policy Service

   Contributor

   More actions

   Tagging subscribers to this area: @agocke, @dotnet/ilc-contrib
   See info in area-owners.md if you want to be subscribed.
5. MichalStrehovsky commented on Sep 30, 2026

   

   MichalStrehovsky

   on Sep 30, 2026

   MemberAuthor

   More actions

   Root cause: ARM64 stackalloc falsely classified as an epilog

   The assertion is caused by a false-negative unwindability classification in NativeAOT’s ARM64 epilog detector. A legitimate stack-allocation instruction is mistaken for an epilog instruction. The interrupted location is marked GC-interruptible, so this produces the contradiction asserted in Thread::HijackCallback.

   Investigated artifacts

   • Checkout: 49e04fa0e472453f4391c4a7097d9e55380dbef2
   • Executable: System.Runtime.Tests, Linux ARM64 NativeAOT
   • Symbols: System.Runtime.Tests.dbg
   • Core: core.1000.17
   • Executable/symbol ELF BuildID: 89b89ce46b63f335c7a0779a423d40ee35b33b88

   All source locations below refer to that checkout.

   1. Interrupted instruction and its purpose

   The aborting thread is OS thread 26. Its relevant stack is:

   RhFailFast
   Assert
   Thread::HijackCallback                 thread.cpp:709
   ActivationHandler
   <signal trampoline>
   String.Concat(IEnumerable<string>)    String.Manipulation.cs:188
   DoubleTests.SplitPairs                DoubleTests.cs:550
   DoubleTests.ParsePatterns             DoubleTests.cs:587
   

   Another thread is in ThreadStore::SuspendAllThreads / GCToEEInterface::SuspendEE.

   The interrupted PC is:

   Runtime address:   0x0000b90f5bdafe04
   ELF virtual addr:  0x00000000022dfe04
   Symbol:            String__Concat_5 + 0xc4
   Function start:    0x0000b90f5bdafd40
   

   Relevant instructions, with offsets relative to the function start:

   +0xac  340006a0  cbz  w0, ...
   +0xb0  a8e07fe0  ldp  x0, xzr, [sp], #-0x200
   +0xb4  910003e0  mov  x0, sp
   +0xb8  f81b83bf  stur xzr, [x29, #-0x48]
   +0xbc  f81c83a0  stur x0,  [x29, #-0x38]
   +0xc0  52802000  mov  w0, #0x100
   +0xc4  b81d03a0  stur w0,  [x29, #-0x30]   // interrupted here

   This corresponds to:

   var result = new ValueStringBuilder(stackalloc char[StackallocCharBufferSizeLimit]);

   in src/libraries/System.Private.CoreLib/src/System/String.Manipulation.cs:188.

   The ldp probes the stack and decreases SP by 512 bytes, allocating the buffer. It does not restore FP or LR. This is deliberate code generation in CodeGen::genLclHeap, src/coreclr/jit/codegenarm64.cpp:3354–3362, which emits:

   ldp tmpReg, xzr, [sp], #-amount
   

   for suitable constant-sized stack allocations.

   2. Exact incorrect classification

   In src/coreclr/nativeaot/Runtime/unix/UnixNativeCodeManager.cpp:1169–1177, TrailingEpilogueInstructionsCount contains:

   // Post-index restore sequences such as "ldp x19, x20, [sp], #0x10" also adjust SP
   // before the final AUTIASP/RET. We avoid signing with a partially-restored SP.
   int baseRegister = (instr >> 5) & 0x1f;
   if (baseRegister == 31)
   {
       if ((instr & LDP_MASK2) == LDP_BITS2 ||
           (instr & LDR_MASK2) == LDR_BITS2)
       {
           return -1;
       }
   }

   The backward scan from the interrupted PC reaches the allocation instruction at +0xb0. For its encoding:

   instruction                          = 0xa8e07fe0
   base register                        = 31 (SP)
   destination registers                = 0 and 31 (x0 and xzr)
   instruction & LDP_MASK2               = 0xa8e07fe0 & 0x7e400000
                                        = 0x28400000
   LDP_BITS2                            = 0x28400000
   signed post-index adjustment         = -512 bytes
   

   Therefore, the function returns -1, despite this being an allocation in the method body.

   On Linux ARM64, IsUnwindable returns whether TrailingEpilogueInstructionsCount(...) == 0 (UnixNativeCodeManager.cpp:592–612). It consequently returns false.

   The matching executable’s disassembly contains this check; this is not merely a discrepancy inferred from the checkout.

   3. The interrupted location is genuinely GC-interruptible

   An independent decoding of the executable’s version-5 GC metadata, following the repository’s GC decoder format, produced:

   FDE ELF virtual address:      0x19b2be0
   LSDA ELF virtual address:     0xbd6f7
   GC-info ELF virtual address:  0xbd6fc
   
   GC header flags:              0x44
   Stack base register:          x29
   Call-site safepoints:         none
   Interruptible ranges:
       [0x02c, 0x1d4)
       [0x1e8, 0x214)
       [0x228, 0x28c)
       [0x2a0, 0x2b8)
       [0x2c8, 0x2e0)
   
   Interrupted offset:           0x0c4
   IsInterruptible:              true
   

   Thus this is an interruptible method-body location, not a call-return safepoint accidentally interpreted as one.

   UnixNativeCodeManager::IsSafePoint accepts decoder.IsInterruptible() as well as decoder.IsSafePoint() (UnixNativeCodeManager.cpp:399–423). The resulting combination is:

   IsSafePoint(interruptedPC)  == true
   IsUnwindable(interruptedPC) == false
   

   That directly explains the assertion in thread.cpp:709.

   4. The frame has usable unwind information

   The FDE describes the established frame as:

   CFA = FP + 16
   saved FP = [CFA - 16]
   saved LR = [CFA - 8]
   

   The core contains:

   FP                = 0x0000ff9458ce4e60
   SP                = 0x0000ff9458ce4c10
   CFA / caller SP   = 0x0000ff9458ce4e70
   [FP + 8]          = 0x0000b90f5c20a9c8
   

   The saved LR resolves to DoubleTests.SplitPairs+136, the expected caller. The dynamic SP allocation does not invalidate this FP-based frame description.

   Importantly, the observed failure is the classification assertion, not an actual failed attempt by the runtime unwinder. The runtime unwinder was not executed experimentally during this investigation.

   5. Introducing change

   The rejecting block was added by:

   c1d33fc8c2966f0a68903c1b8b1f6c462e1b3a96
   Arm64: [PAC-RET] NativeAOT changes
   dotnet/runtime#128950
   July 9, 2026, 01:07:37 UTC
   

   Evidence beyond blame:

   • The commit’s parent lacks the SP-only load rejection.
   • The allocation code and assertion already existed before that commit.
   • Replaying the detector against this executable’s instruction words gives:

   Current detector:
       scan +0xc0, +0xbc, +0xb8, +0xb4
       encounter +0xb0: SP-based LDP
       return -1 → not unwindable
   
   Pre-change detector:
       scan through +0xb0: neither FP nor LR is restored
       encounter +0xac: CBZ branch boundary
       return 0 → unwindable
   

   This establishes the source-level introduction of the bad predicate. It is not a historical binary bisection or a determination of the first failing CI run.

   6. Fix considerations

   The fix belongs in the epilog/unwindability classification, not in String.Concat, the test, or removal of the assertion.

   Additional concerns identified during source inspection—not separate failures observed in this core:

   • The SP-load masks also accept non-writeback loads. Merely recognizing an SP-based load does not establish that stack restoration occurred.
   • Those same masks serve legitimate FP/LR-restore recognition elsewhere. Globally narrowing them could break actual epilog detection.
   • The separate new ADD SP,SP rejection at UnixNativeCodeManager.cpp:1181–1186 can also match ordinary localloc code: codegenarm64.cpp:3309–3325 temporarily removes the outgoing-argument area using that instruction.
   • Gating the checks on PAC alone would not eliminate false positives in PAC-enabled method bodies.

   A direction worth evaluating is to keep precise GC unwindability detection separate from additional PAC-specific hijack restrictions, while retaining necessary handling of AUTIASP/AUTIBSP. No exact patch has been validated, and deleting all new SP checks has not been established as safe.

   Reproducing the debugger observations

   With the executable, symbols, and core in the current directory:

   lldb -c core.1000.17 System.Runtime.Tests
   

   Then:

   thread select 1
   bt 11
   frame select 8
   register read pc sp fp lr
   disassemble -s 0xb90f5bdafdec -c 7 --bytes
   image show-unwind -a 0xb90f5bdafe04
   memory read -s 8 -f x -c 2 0xff9458ce4e60
   image lookup -a 0xb90f5c20a9c8
   disassemble -n "UnixNativeCodeManager::IsUnwindable"
   

   These addresses and frame numbers are specific to the supplied dump. Executable text omitted from the core is supplied by the matching executable.

   Validation scope: matching core/binary inspection, instruction-level predicate replay, independent GC-metadata decoding, CFI/saved-frame inspection, and source-history comparison. No source changes, product rebuild, runtime reproduction, or candidate-fix validation were performed.

   Note

   This investigation write-up was generated with GitHub Copilot, including an independent GPT-6 Astra challenge of the diagnosis.
6. MichalStrehovsky commented on Sep 30, 2026

   

   MichalStrehovsky

   on Sep 30, 2026

   MemberAuthor

   More actions

   Cc @SwapnilGaikwad seems to caused by the PAC-RET change.

   👍React with 👍1Reacted by SwapnilGaikwad
7. jkotas

   added

   regression-from-last-release

   tenet-reliabilityReliability/stability related issue (stress, load problems, etc.)Reliability/stability related issue (stress, load problems, etc.)

   on Oct 1, 2026
8. SwapnilGaikwad

   self-assigned this

   on Oct 1, 2026
9. SwapnilGaikwad commented on Oct 1, 2026

   

   SwapnilGaikwad

   on Oct 1, 2026

   Contributor

   More actions

   Treating SP adjustment instructions being part of epilog doesn't seem to be a good idea. These additional instruction checks helped to bail out of hijacking conservatively (but seemingly incorrectly!).
   As a temporary fix we can gate checks for these instructions during hijacking, i.e., when TrailingEpilogueInstructionsCount() is called from GetReturnAddressHijackInfo(). It would let other places where TrailingEpilogueInstructionsCount() called rely on the original mechanism and hijacking would be more conservative.
   A cleaner fix would be probably to rely on signing SP == CFA and retrieve signing SP accordingly for NativeAOT on non-windows. This will allow us to restore original TrailingEpilogueInstructionsCount() logic and retrieving singing sp would not depend on having stack adjustment instructions in function body . Any thoughts?
10. jkotas commented on Oct 2, 2026

    

    jkotas

    on Oct 2, 2026

    Member

    More actions

    It would let other places where TrailingEpilogueInstructionsCount() called rely on the original mechanism and hijacking would be more conservative.

    The other places are only used for debug checks. If we see this assert firing too often in CI, the easiest way to fix that is by commenting it out and accepting that the IsUnwindable check can give false negatives on arm64 like it does on other plaforms:

    runtime/src/coreclr/nativeaot/Runtime/thread.cpp

    Lines 701 to 704 in 32314b9

    	// IsUnwindable is precise on arm64, but can give false negatives on other architectures.
    	// (when IP is on the first instruction of an epilog, we still can unwind,
    	// but we can tell if the instruction is the first only if we can navigate instructions backwards and check)
    	// The preciseness of IsUnwindable is tracked in https://github.com/dotnet/runtime/issues/101932

    A cleaner fix would be probably to rely on signing SP == CFA and retrieve signing SP accordingly for NativeAOT on non-windows.

    I am not sure how it would work. Can you elaborate?

    Another way to address this would be to encode the prologs/epilogs in GCInfo as suggested in #101932 .
11. SwapnilGaikwad commented on Oct 2, 2026

    

    SwapnilGaikwad

    on Oct 2, 2026

    Contributor

    More actions

    I am not sure how it would work. Can you elaborate?

    Another way to address this would be to encode the prologs/epilogs in GCInfo as suggested in #101932 .

    I'll take a look at #101932. I've created a draft patch that uses CFA as singing SP for NativeAOT. Also put an assert checking this in unwindarm64.cpp. Could you take a look and see if that's reasonable, please?
12. jkotas

    added

    Known Build ErrorUse this to report build issues in the .NET Helix tabUse this to report build issues in the .NET Helix tab

    blocking-clean-ci-optionalBlocking optional rolling runsBlocking optional rolling runs

    on Oct 4, 2026
13. SwapnilGaikwad

    added a commit that references this issue on Oct 5, 2026

    Arm64:[PAC-RET]Fix ARM64 epilog detection and PAC signing SP recovery

    ...

    This issue will close once commit d7ccf69 is merged into the 'main' branch.

    d7ccf69
14. jkotas

    closed this as completedin #135088on Oct 6, 2026
15. dotnet-policy-service

    removed

    untriagedNew issue has not been triaged by the area ownerNew issue has not been triaged by the area owner

    on Oct 6, 2026
16. jkotas

    added a commit that references this issue on Oct 6, 2026

    Arm64:[PAC-RET]Fix ARM64 epilog detection and PAC signing SP recovery (…

    ...

    This issue will close once commit 4cc1f5a is merged into the 'main' branch.

    4cc1f5a