Repository navigation
Stack trace line numbers wrong after tier-1. Possible related to PGO/GDV #134523
Description
Description
We have noticed than since switching from .NET Framework to .NET Core many of our stack traces will always point to the first line of a method, even if the exception happens several lines later.
While worse stack traces might be a side-effect of compiler optimizations, investigating the method with WinDbg/SOS indicates that the runtime is indeed aware of the correct line number.
Reproduction Steps
public class Repro
{
[MethodImpl(MethodImplOptions.NoInlining)]
public bool Method(Func<IHolder> holderFactory, bool flag)
{
var holder = holderFactory();
return flag && holder.Data.ToString() != null; // NullReferenceException, since Data is null
}
}
public interface IHolder
{
object Data { get; }
}
class HolderImpl : IHolder
{
public object Data { get; set; }
}
internal class Program
{
public static void Main(string[] args)
{
var r = new Repro();
// Before tier-1
try
{
r.Method(() => new HolderImpl(), true);
}
catch (NullReferenceException e)
{
Console.WriteLine(e);
}
// Warm-up
for (int i = 0; i < 1000000; i++)
{
try
{
r.Method(() => new HolderImpl(), true);
}
catch (NullReferenceException) { }
}
// After tier-1
try
{
r.Method(() => new HolderImpl(), true);
}
catch (NullReferenceException e)
{
Console.WriteLine(e);
}
}
}
Expected behavior
Both exceptions should show NRE happened on line 10
System.NullReferenceException: Object reference not set to an instance of an object.
at Repro.Method(Func1 holderFactory, Boolean flag) in C:\projects\WrongStackRepro\WrongStackRepro\Program.cs:line 10 at Program.Main(String[] args) in C:\projects\WrongStackRepro\WrongStackRepro\Program.cs:line 33 System.NullReferenceException: Object reference not set to an instance of an object. at Repro.Method(Func1 holderFactory, Boolean flag) in C:\projects\WrongStackRepro\WrongStackRepro\Program.cs:line 10
at Program.Main(String[] args) in C:\projects\WrongStackRepro\WrongStackRepro\Program.cs:line 53
Actual behavior
Exception after tier-1 shows NRE happened in first line of method (line 8)
System.NullReferenceException: Object reference not set to an instance of an object.
at Repro.Method(Func1 holderFactory, Boolean flag) in C:\projects\WrongStackRepro\WrongStackRepro\Program.cs:line 10 at Program.Main(String[] args) in C:\projects\WrongStackRepro\WrongStackRepro\Program.cs:line 33 System.NullReferenceException: Object reference not set to an instance of an object. at Repro.Method(Func1 holderFactory, Boolean flag) in C:\projects\WrongStackRepro\WrongStackRepro\Program.cs:line 8
at Program.Main(String[] args) in C:\projects\WrongStackRepro\WrongStackRepro\Program.cs:line 53
Regression?
Does not happen in .NET 7, happens in .NET 8 and later
Known Workarounds
Adding [MethodImpl(MethodImplOptions.AggressiveOptimization)] (thus disabling PGO) will make the issue disappear, hence why I suspect the issue might be related to PGO/GDV.
Configuration
Windows, x64, .NET10
Other information
When viewing method in WinDbg it correctly maps the assembly instruction that is causing the NRE to the correct source line number:
0:009> !U /d 00007ffd9b6ebc00
Normal JIT generated code
Repro.Method(System.Func`1<IHolder>, Boolean)
ilAddr is 000001F10EAA2050 pImport is 000002AB23B46490
Begin 00007FFD9B6EBC00, size 7d
C:\projects\WrongStackRepro\WrongStackRepro\Program.cs @ 8:
>>> 00007ffd`9b6ebc00 53 push rbx
00007ffd`9b6ebc01 4883ec20 sub rsp,20h
00007ffd`9b6ebc05 418bd8 mov ebx,r8d
00007ffd`9b6ebc08 48b980228b9bfd7f0000 mov rcx,offset Program+<>c.<Main>b__0_3() (00007ffd`9b8b2280)
00007ffd`9b6ebc12 48394a18 cmp qword ptr [rdx+18h],rcx
00007ffd`9b6ebc16 753f jne WrongStackRepro!Repro.Method(System.Func`1<IHolder>, Boolean)+0x6247 (00007ffd`9b6ebc57)
00007ffd`9b6ebc18 48b978208f9bfd7f0000 mov rcx,7FFD9B8F2078h (MT: HolderImpl)
00007ffd`9b6ebc22 e81996c25f call coreclr!coreclr_set_error_writer+0x5d20 (00007ffd`fb315240) (JitHelp: CORINFO_HELP_NEWSFAST)
C:\projects\WrongStackRepro\WrongStackRepro\Program.cs @ 10:
00007ffd`9b6ebc27 84db test bl,bl
00007ffd`9b6ebc29 744a je WrongStackRepro!Repro.Method(System.Func`1<IHolder>, Boolean)+0x6265 (00007ffd`9b6ebc75)
00007ffd`9b6ebc2b 48b978208f9bfd7f0000 mov rcx,7FFD9B8F2078h (MT: HolderImpl)
00007ffd`9b6ebc35 483908 cmp qword ptr [rax],rcx
00007ffd`9b6ebc38 7526 jne WrongStackRepro!Repro.Method(System.Func`1<IHolder>, Boolean)+0x6250 (00007ffd`9b6ebc60)
00007ffd`9b6ebc3a 488b4808 mov rcx,qword ptr [rax+8]
00007ffd`9b6ebc3e 488b01 mov rax,qword ptr [rcx]
00007ffd`9b6ebc41 488b4040 mov rax,qword ptr [rax+40h]
00007ffd`9b6ebc45 ff5008 call qword ptr [rax+8]
00007ffd`9b6ebc48 4885c0 test rax,rax
00007ffd`9b6ebc4b 0f95c0 setne al
00007ffd`9b6ebc4e 0fb6c0 movzx eax,al
00007ffd`9b6ebc51 4883c420 add rsp,20h
00007ffd`9b6ebc55 5b pop rbx
00007ffd`9b6ebc56 c3 ret
C:\projects\WrongStackRepro\WrongStackRepro\Program.cs @ 8:
00007ffd`9b6ebc57 488b4a08 mov rcx,qword ptr [rdx+8]
00007ffd`9b6ebc5b ff5218 call qword ptr [rdx+18h]
00007ffd`9b6ebc5e ebc7 jmp WrongStackRepro!Repro.Method(System.Func`1<IHolder>, Boolean)+0x6217 (00007ffd`9b6ebc27)
C:\projects\WrongStackRepro\WrongStackRepro\Program.cs @ 10:
00007ffd`9b6ebc60 488bc8 mov rcx,rax
00007ffd`9b6ebc63 49bbe8005e9bfd7f0000 mov r11,7FFD9B5E00E8h
00007ffd`9b6ebc6d 41ff13 call qword ptr [r11] (CLRStub[VSD_LookupStub]@00007FFD9B6E04C0)
00007ffd`9b6ebc70 488bc8 mov rcx,rax
00007ffd`9b6ebc73 ebc9 jmp WrongStackRepro!Repro.Method(System.Func`1<IHolder>, Boolean)+0x622e (00007ffd`9b6ebc3e)
00007ffd`9b6ebc75 33c0 xor eax,eax
00007ffd`9b6ebc77 4883c420 add rsp,20h
00007ffd`9b6ebc7b 5b pop rbx
00007ffd`9b6ebc7c c3 ret
The line that is causing the NRE:
00007ffd9b6ebc3e 488b01 mov rax, qword ptr [rcx]
is shown as part of a Program.cs @ 10 block.
Activity
dotnet-policy-service commentedon Sep 23, 2026
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.
dotnet-policy-service commentedon Sep 23, 2026
Tagging subscribers to this area: @steveisok, @tommcdon, @dotnet/dotnet-diag
See info in area-owners.md if you want to be subscribed.
EgorBo commentedon Sep 23, 2026
From a quick AI triage, it looks like VM off-by-one in stack-trace mapping, not incorrect GDV codegen
anderspedersen commentedon Oct 2, 2026
@EgorBo
I did a bit of digging, and I believe issue is partly caused by shortcomings in the GDV codegen.
Basically there seems to be two issues at play for my repro:
Stack trace mapping seems to assume that IPs are return addresses, which is true when IL_Throw is called, but false when exception is caused by CPU. So it tries to find the preceding instruction, in my repro it means it is using the mapping from the devirtualized call to get_Data, rather than the mapping from the instruction that caused the NRE.
The mapping from devirtualized calls seems to be missing in some cases. Tried to build latest runtime and get following IP mapping for my repro:
With codegen:
So, NRE is happening at offset 0x4B, but is using mapping from offset 0x47, that is the devirtualized and inlined call to HolderImpl.get_Data and has no mapping.
Notice that we are also seeing same issue from software exceptions, so the regression since older .NET versions seems to be the missing IP mapping, rather than the wrong stack trace mapping.
I tried fixing the IP mapping issue, by letting the basic blocks generated by GDV get their internal status based on the basic block that is being expanded. This fixes the issue, but don't know if that is the direction you want to go. If so, let me know, and I will be happy to create a PR.
anderspedersen@fa8c130
I have attached JitDumps from before my change and after:
jitdump_before_fix.txt
jitdump_fixed.txt
anderspedersen commentedon Oct 4, 2026
I did some more debugging, and I believe my original fix was wrong, not marking the basic block as internal was just suppressing the NO_MAP mappings, but not actually inserting the correct mappings.
As far as I can tell, the issue happens because when doing GDV of a method with an return value, the compiler inserts a statement for the call with DebugInfo and a statement for the return expression without DebugInfo. Later, more logic from the devirtualized call is merged into the return expression statement, that remains without DebugInfo.
I tried patching the code to include DebugInfo with the return expression statement. This seems to fix the issue.
anderspedersen@0fce089
BB08 right after devirtualization:
***** BB08 [0007] STMT00012 ( ??? ... ??? ) [000044] DA--------- * STORE_LCL_VAR ref V08 tmp4 [000043] ----------- \--* LCL_VAR ref V03 loc0 ***** BB08 [0007] STMT00013 ( 0x00A[E--] ... ??? ) - [000045] I-CXG------ * CALL nullcheck ref HolderImpl:get_Data():System.Object:this (exactContextHandle=0x00007FF9250450D9) + [000045] I-CXG------ * CALL nullcheck ref HolderImpl:get_Data():System.Object:this (exactContextHandle=0x00007FF92ACC4F91) [000047] ----------- this \--* LCL_VAR ref V08 tmp4 ***** BB08 [0007] -STMT00014 ( ??? ... ??? ) +STMT00014 ( 0x00A[E--] ... ??? ) [000049] DAC-------- * STORE_LCL_VAR ref V07 tmp3 [000048] --C-------- \--* RET_EXPR ref (for [000045])And BB08 after optimizations:
And the final IP mappings: