Fix ilasm PdbChecksum and the PDB ID under -DET by pcshrosbree · Pull Request #135212 · dotnet/runtime

pcshrosbree · 2026-10-05T13:18:38Z

ilasm's PdbChecksum debug directory entry hashes only the #Pdb stream, whereas PE-COFF.md requires a hash of the whole PDB file with its 20-byte ID zeroed, so the checksum never verifies. With -DET, the PDB ID's stamp also stays 0 while the CodeView entry carries the deterministic stamp, so PEReader.TryOpenAssociatedPortablePdb rejects the pair.

Change

PortablePdbWriter::ComputeSha256PdbChecksum replaces ComputeSha256PdbStreamChecksum. It zeroes the PDB ID in the #Pdb heap, serializes the PDB with GetSaveSize and SaveToMemory, hashes the result and restores the ID. At that point the PDB is final apart from its ID, and Assembler::SavePdbFile later saves it through the same emitter.
IILAsmPortablePdbWriter::ChangePdbStreamGuid becomes ChangePdbStreamId(guid, stamp). It is backed by PdbHeap::SetDataId, which writes all 20 bytes with the same big-endian handling as PdbHeap::SetData. -DET uses it to write both fields.
The now-unused ComputeSha256PdbStreamChecksum and PdbHeap::ComputeSha256Checksum are removed.
TestPortablePdbDebugDirectory now runs with and without -DET. It compares all 20 bytes of the PDB ID with the CodeView entry (before, it compared only the GUID), and checks the PdbChecksum entry against SHA-256 of the PDB file with its ID zeroed.

Notes

The output changes only where it was wrong. Assembling the same IL with the base and the fixed ilasm in one directory, a -DET PDB differs only in its 4 stamp bytes, and the DLL only in its 32 checksum bytes. So -DET output is not byte-identical to an older ilasm's.
The PDB is now serialized twice. On a 552 KB PDB (2,500 methods, 200 documents), the difference was within noise: a median of 48 ms against 49 ms per ilasm run, over 5 runs each.
The IILAsmPortablePdbWriter vtable changes but its IID does not. ilasm is the interface's only consumer, and it links mdcompiler_ppdb statically. I can give the interface a new IID if you prefer.

Validation (Linux x64)

ilasm_tests against a Checked Core_Root, with counts taken from ilasm_tests.testResults.xml:

ilasm	Result
base	4 of 15 fail: the two non-`-DET` cases on the checksum, and the two `-DET` cases on the stamp
fixed	15 of 15 pass

Each new assertion was also checked to fail with the corresponding part of the fix reverted.

Nine IL inputs were assembled with and without -DET, covering no sequence points, no methods, several documents with an entry point, a 552 KB PDB, /DEBUG=OPT, /DEBUG=IMPL and /PDB. On every output:

the checksum matches only the PE-COFF.md rule;
the PDB ID matches the CodeView entry;
TryOpenAssociatedPortablePdb succeeds.

Two -DET runs in the same directory give identical bytes.

Not run: Windows, macOS, Arm64 or a big-endian target (the BIGENDIAN branch of SetDataId is compiled out on x64), and the full src/tests suite.

Resolves #135211

Note

This change, its validation and this description were prepared with AI assistance (Anthropic Claude and OpenAI Codex) under my direction. AI agents ran the builds, tests and probes on my machine. I reviewed the diff, the results and this text before posting.

ilasm computed the PdbChecksum debug directory entry over the #Pdb stream only, while PE-COFF.md requires the hash of the entire PDB file with its 20-byte PDB ID zeroed. Under -DET it also wrote only the deterministic GUID into the #Pdb stream, so the PDB ID's timestamp stayed 0 while the CodeView entry carried the deterministic stamp, and PEReader.TryOpenAssociatedPortablePdb rejected the pair. Compute the checksum by serializing the PDB with its ID temporarily zeroed, and write both the GUID and the timestamp into the #Pdb stream. Extend TestPortablePdbDebugDirectory to compare all 20 bytes of the PDB ID with the CodeView entry and to verify the PdbChecksum entry, with and without -DET.

azure-pipelines · 2026-10-05T13:19:35Z

Azure Pipelines: Successfully started running 4 pipeline(s). 12 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

dotnet-policy-service · 2026-10-05T13:19:51Z

Tagging subscribers to this area: @JulieLeeMSFT, @dotnet/jit-contrib
See info in area-owners.md if you want to be subscribed.

jkotas · 2026-10-05T16:45:25Z

@pcshrosbree ilasm is being rewritten in C#. C/C++ implementation of ilasm is going to be deleted soon.

You may want to check whether the issues you are running into are fixed in the C# rewrite https://github.com/dotnet/runtime/tree/main/src/tools/ilasm , and if not - fix them there instead.

pcshrosbree · 2026-10-05T17:33:43Z

Thanks for the pointer. Here is how the managed ilasm at main (b847b8718d0, Release, Linux x64) handles both issues, with the inputs from #135214:

ilasm: -DET derives the MVID, PE timestamp and PDB ID from a zero-filled buffer, so different assemblies share them #135214 does not reproduce. With -DET, the five inputs get five distinct MVIDs and PE timestamps, with and without -DEBUG, and repeated outputs are byte-identical. PDB IDs follow the PDB's content: the two .line variants get different IDs, and inputs whose PDBs are identical share one. The MVID and timestamp match BlobContentId.FromHash of the image's SHA-256 with those fields zeroed. The PDB ID matches the same function applied to the PDB with its ID zeroed.
Neither defect from ilasm: PdbChecksum hashes only the #Pdb stream, and -DET leaves the PDB ID's stamp at 0 #135211 appears. No PdbChecksum entry is written, and the CodeView GUID and stamp match the embedded PDB's ID.

A few PDB differences from native ilasm I'd be happy to follow up on, if they're unintended:

-DEBUG and -PDB embed the PDB, where native ilasm writes a separate file (Min.pdb for Min.dll). The tests expect embedding, but the -PDB help text still reads "Create the PDB file without enabling debug info tracking". .line directives also produce an embedded PDB without either switch; native ilasm needs -DEBUG or -PDB.
The CodeView entry always names assembly.pdb. Should it use the output's name, such as Min.pdb for Min.dll, for tools that look up a separate or extracted PDB?

Would the native fixes be useful for servicing branches that still ship native ilasm? If not, I'll close this PR and #135227, and leave #135211 and #135214 for you to dispose of.

Note

This check and this reply were prepared with AI assistance (Anthropic Claude and OpenAI Codex) under my direction. AI agents ran the build and the checks on my machine. I reviewed the text before posting.

jkotas · 2026-10-05T22:24:28Z

A few PDB differences from native ilasm I'd be happy to follow up on, if they're unintended:

@jkoritzinsky Could you please comment on this?

jkoritzinsky · 2026-10-06T12:08:58Z

Those changes are definitely unintended and should be fixed in managed ilasm.

pcshrosbree · 2026-10-06T13:40:15Z

Thanks @jkoritzinsky. I'm working on the follow-ups for the three PDB differences in managed ilasm:

-DEBUG and -PDB write the PDB as a separate file beside the output (Min.pdb for Min.dll), with a PdbChecksum entry alongside the CodeView entry, instead of embedding it;
the CodeView entry names that file rather than assembly.pdb;
.line directives alone no longer produce a PDB; one is written only with -DEBUG or -PDB, as in native ilasm.

PR to follow against main, with tests and a note in the ilasm README describing the behaviour. I'll leave this PR and #135227 open until you say whether the native fixes are wanted for servicing.

am11 · 2026-10-06T13:50:53Z

for servicing

Our bar for servicing typically depends on whether a bug is a regression from a previous release. Because this is a long-standing behavior, and we are currently rewriting the managed ilasm (followed by ildasm), it would be best to combine our efforts toward completing those rewrites so they can serve as a drop-in replacement for the native implementation before old code is deleted.

pcshrosbree · 2026-10-06T14:01:04Z

Thanks @am11. To make sure I read this right: should I close this PR and #135227, and leave #135211 and #135214 to you?

Some context on why I opened them: my current project is hit by both defects in the native ilasm that ships for .NET 8 and .NET 10 (a PdbChecksum entry that does not verify, with a CodeView id that does not match the PDB's, and -DET giving different inputs the same identity), which is what prompted the PR. Is the managed ilasm likely to replace the native one in the .NET 8 or .NET 10 packages in the next few weeks, or should I plan to work around these issues in my project for those releases?

The managed-ilasm PR for the three PDB differences above is in review on my side and will go up shortly.

pcshrosbree · 2026-10-06T16:43:29Z

@jkoritzinsky, the managed ilasm fix is up in #135289. With -DEBUG or -PDB the PDB is now written to <output>.pdb beside the output, and .line alone no longer produces one. The CodeView entry names the PDB's full path, as in native ilasm, so a -DET image depends on its output directory. It is followed by PdbChecksum and, with -DET, Reproducible. The PR also adjusts the IL round-trip harness, whose PDB determinism step relied on the PDB overwriting the image.

Note

This change was prepared with AI assistance (Anthropic Claude and OpenAI Codex) under my direction. AI agents wrote the code and ran the build and tests on my machine. I reviewed the change before posting.

pcshrosbree · 2026-10-06T20:56:05Z

For anyone following the managed ilasm's PDB output from here: the differences from native ilasm that remain after #135289, #135297, #135311 and #135312 are listed in #135314, grouped so they can be picked up one at a time. Nothing beyond those PRs is planned unless a maintainer names it there.

@jkotas

…135289) In #135212, @jkotas [pointed out](#135212 (comment)) that native ilasm is being replaced by the C# rewrite in `src/tools/ilasm` and suggested fixing issues there. I [listed](#135212 (comment)) three PDB differences between the managed and native ilasm, and @jkoritzinsky [confirmed](#135212 (comment)) that they are unintended and should be fixed in managed ilasm. This PR fixes them. ### Changes | | Before | After | |---|---|---| | `-DEBUG` / `-PDB` | PDB embedded in the image (`EmbeddedPortablePdb` entry) | PDB written to `<output>.pdb` beside the output, not embedded, as native ilasm | | `.line` without `-DEBUG` or `-PDB` | embedded PDB produced | no PDB and no debug directory, as native ilasm; `.line` is still parsed and validated | | CodeView path | always `assembly.pdb` | full path of the PDB file, as native ilasm; under `-DET` only its file name (`Min.pdb`), a deliberate difference that follows the linker's `/PDBALTPATH:%_PDB%` | - **Debug directory.** It is CodeView, then `PdbChecksum`, then, with `-DET`, `Reproducible`: the same entry types in the same order as native ilasm. - The checksum is SHA-256 of the PDB with its 20-byte id zeroed (PE-COFF.md). - It is the hash the `PortablePdbBuilder` id provider computes, so under `-DET` the PDB id and the checksum come from one hash. - **No PDB, no debug directory.** An image built without a PDB has no debug directory, with or without `-DET`. `ManagedPEBuilder` adds a Reproducible-only directory to a deterministic image when it is given none. A small `ILAssemblerPEBuilder` subclass, which the vtfixup/export builder also derives from, clears a debug directory with no entries. - **Library and CLI.** PDB generation does not touch the file system. - `CompilationResult.PortablePdb` returns the PDB bytes, the new `Options.PdbFilePath` supplies the CodeView path, and the CLI writes the file. - Library callers that don't set `PdbFilePath` get the output file name with a `.pdb` extension, or `assembly.pdb`. - **File output** is split in two: `OutputWriter` in `ILAssembler` works on streams and makes the decisions below, and `OutputFileWriter` in `ilasm` supplies the files: - **Write order.** It writes and closes the image, then writes the PDB to a new temporary file in the same directory and renames it over `<output>.pdb`, so `<output>.pdb` never holds a partial PDB. If the PDB cannot be written, the run fails with the new image beside the previous PDB, and deleting the temporary file is attempted. - **Stale PDBs.** After writing an image without a PDB, it deletes an existing `<output>.pdb` only if it belongs to the image that was replaced, that is, if the previous image's CodeView entry names that PDB's id. Any other file is kept, including when the previous output is not a PE image it can read (a COFF object file or a file over 2 GB, for example), and an output that is itself named like its PDB is never deleted. A PDB that cannot be deleted is left in place without failing the run. When assembly fails and produces no output, it writes and deletes nothing; with `-ERR`, an image produced despite errors is written and its PDB is handled as for a successful assembly. - Native ilasm deletes `<output>.PDB` whenever it writes no PDB, including when assembly fails. I dropped that deliberately, because it can remove a PDB that ilasm did not write or that belongs to an image that was not replaced. - On Linux native deletes `<output>.PDB`, so it has not been removing `.pdb` files there. - **Name collisions.** If the PDB path would be the output path (`-OUTPUT=Min.pdb -DEBUG`), it reports an error and writes nothing, where native ilasm overwrites the image with the PDB. The comparison ignores case on Windows and macOS. - **Switches.** `--debug-mode` alone produces a PDB, as it already implied `--debug` for the `DebuggableAttribute`. `-PDB` still adds no `DebuggableAttribute` of its own. - **Docs.** - The options have doc comments. - The `DebugMode.Impl` docs said it enables Edit and Continue. IMPL is 0x103, `Default | IgnoreSymbolStoreSequencePoints | DisableOptimizations`, and the docs now say so. - The `--debug-mode` help says what the modes do to the JIT. **Round-trip harness (needed for this PR to merge).** - `src/tests/Common/CLRTest.Jit.targets` checked PDB determinism by assembling with `-DEBUG -output=<name>.pdb` and hashing that file. That works with native ilasm only because the PDB overwrites the image. - Managed ilasm now refuses that output path, so the `managedilasmroundtrip` lane would fail. - The third commit assembles that step to `IL-RT/<name>.pdbcheck.dll` and hashes both that image and the `.pdb` beside it, which both ilasms write. The previous managed run had also checked the `-DEBUG` image, by hashing it with the PDB embedded. - I ran the changed step by hand with native and managed ilasm. Both repeat byte-identically, and a run without `-DET` fails the image comparison, and with the image comparison removed fails the PDB comparison, as it should. I could not run the lanes themselves. In the round-trip lane, reassembling a non-debug test assembly without `-DEBUG` now removes the compiler's `<name>.pdb`, which belongs to the replaced image; the reassembled image has no debug directory, so nothing reads it, and the previous step overwrote that file anyway. **Determinism.** With `-DET` the CodeView entry records only the PDB's file name, so the same input assembled to outputs of the same name in different directories gives byte-identical images and PDBs (native ilasm records the full path in both modes; this follows the linker's `/PDBALTPATH:%_PDB%` convention instead). **Remaining differences from native ilasm:** - PdbChecksum entry timestamp: 0 in managed (`DebugDirectoryBuilder`), the PDB stamp in native. The managed entry is new in this PR. - Nondeterministic PDB stamp: the constant `0x04030201` in managed, the clock in native. - `-DET` PDB id: the PDB content hash in managed, the metadata hash in native. - `-OUTPUT=a.b/Min`: native writes `a.pdb` in the parent directory; managed writes `a.b/Min.pdb`. That naming is deliberate. - CodeView path under `-DET`: the PDB's file name in managed, its full path in native. That is deliberate (see Determinism above). - Stale PDB removal: the ownership rule above, versus native's unconditional `<output>.PDB` deletion. - PDB content: managed has no local scopes, uses one document per method, and does not record the `.il` source as a document. - Netmodules: native emits the `DebuggableAttribute` on `AssemblyAttributesGoHere`; managed emits it only with `.assembly`. ### Testing (Linux x64) - `ILAssembler.Tests`: 1216 before; 2124 at commit 1, 2332 at commits 2 and 3, 2344 at commit 4, 2366 at commits 5 and 6, 2374 at commit 7, and 2379 at commit 8, all passing on Linux; on Windows two file-system tests skip, five symbolic-link tests skip unless the process can create symbolic links, and the relative-output test skips if the temporary directory is on another drive than the working directory. - Unit tests read the PDB from the compilation result. They pin: - the entry order, with and without `-DET`; - the CodeView path, its fallback, and its id; - the checksum, against the PDB with its id zeroed; - no PDB from `.line` alone; - no debug directory or debug data without a PDB, including through the vtfixup/export builder; - `-PDB` without a `DebuggableAttribute`; - deterministic PDB and image bytes. - `OutputWriterTests` tests the output writer on in-memory streams, without the file system. It covers: - the write order, including closing the image before touching the PDB (an injected close failure); - which existing PDB it deletes, including when the replaced output is a COFF object, longer than 2 GB, or cannot be opened; - the refusal to write a PDB over the output. - `OutputFileWriterTests` covers the file side: the PDB path, the temporary file and rename, deletion, and the case rule. - Seeded theories check the same rules over generated programs and option combinations (800 cases). They also check the file writer over an enumeration of output names and pre-existing output and PDB states (270 cases). Each case is named by its index and description. - `CommandLinePdbFileTests` runs the command line in process on IL sources in a temporary directory (27 cases). It covers: - the PDB beside the output and its CodeView entry; - under `-DET`, a CodeView entry with only the PDB's file name, for absolute and relative output paths, and the PDB found beside the image from that name; - the entry order and the checksum; - no PDB without a switch, with `.line` and with `-DET`; - `-PDB` without a `DebuggableAttribute`; - stale-PDB replacement and deletion, and an unrelated PDB kept; - failed builds; - `-ERR` with a recoverable error; - the refusal of an output named like its PDB; - byte-identical `-DET` repeats, and byte-identical `-DET` images and PDBs across two output directories. The cases that ran native ilasm as a control were dropped when these tests moved out of `src/tests/ilasm`. - I ran 59 selected mutations, one at a time: 19 in the compiler (checksum, entry order, gating), 32 in the two output writers and 8 in the command line. 58 were detected. The survivor removes a file-existence check before opening a file; it is equivalent for stable files, because a missing file raises an `IOException` that the library counts as no id. - I ran the built CLI end to end and compared it with native ilasm, both as built from this branch and with #135212's fix. I checked the files written, the entries, the checksum, byte-identical `-DET` repeats, and stale-PDB handling. - `src/tests/ilasm/PortablePdb` against managed ilasm: 0/10 before, 4/10 after. - Both `TestPortablePdbDebugDirectory` cases now pass. - The six failures have three causes, all PDB content this PR does not touch: local scopes (4 cases), a method whose sequence points span two documents (`TestDocuments1`), and the `.il` source as a document (`TestPortablePdbDocuments`). Fixes for those are in progress separately and will follow as their own PRs. Not run: Windows, macOS, Arm64, the round-trip lanes, or the full `src/tests` suite. The ilasm pipeline runs on this PR and covers the round-trip lanes on eight platforms and `ILAssembler.Tests` on Windows x64. Questions: - Is `ImmutableArray<byte>? CompilationResult.PortablePdb` the shape you want, or `ImmutableArray<byte>` with `IsDefault`? - Would you rather have an option in `System.Reflection.Metadata` for "no debug directory" than the `ILAssemblerPEBuilder` subclass? `ManagedPEBuilder` has no public way to decline its default Reproducible entry. - Following the review, the CodeView entry now holds only the PDB's file name under `-DET`, and its full path otherwise. - Is deleting `<output>.pdb` only when it belongs to the replaced image the behaviour you want? - Is the `--debug-mode` help wording right? > [!NOTE] > This change was prepared with AI assistance (Anthropic Claude and OpenAI Codex) under my direction. AI agents wrote the code and ran the build and tests on my machine. I reviewed the change before posting.

pcshrosbree mentioned this pull request Oct 5, 2026

ilasm: PdbChecksum hashes only the #Pdb stream, and -DET leaves the PDB ID's stamp at 0 #135211

Open

github-actions Bot added the area-ILTools-coreclr label Oct 5, 2026

dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Oct 5, 2026

jkotas requested a review from jkoritzinsky October 5, 2026 13:40

build-analysis Bot mentioned this pull request Oct 5, 2026

[ci-scan] Hang: System.Diagnostics.Process.Tests times out on windows-x86 Checked CoreCLR #135001

Open

pcshrosbree mentioned this pull request Oct 5, 2026

Derive ilasm -DET identities from the assembly's content #135227

Open

pcshrosbree mentioned this pull request Oct 6, 2026

Write the managed ilasm PDB to a separate file, as native ilasm does #135289

Merged

Navigation Menu

Repository navigation

Fix ilasm PdbChecksum and the PDB ID under -DET - #135212

Fix ilasm PdbChecksum and the PDB ID under -DET#135212
pcshrosbree wants to merge 1 commit into
dotnet:mainfrom
pcshrosbree:fix/ilasm-pdb-checksum

pcshrosbree commented Oct 5, 2026

Uh oh!

azure-pipelines Bot commented Oct 5, 2026

Uh oh!

dotnet-policy-service Bot commented Oct 5, 2026

Uh oh!

jkotas commented Oct 5, 2026 •

edited

Loading

Uh oh!

pcshrosbree commented Oct 5, 2026

Uh oh!

jkotas commented Oct 5, 2026

Uh oh!

jkoritzinsky commented Oct 6, 2026

Uh oh!

pcshrosbree commented Oct 6, 2026

Uh oh!

am11 commented Oct 6, 2026

Uh oh!

pcshrosbree commented Oct 6, 2026

Uh oh!

pcshrosbree commented Oct 6, 2026

Uh oh!

pcshrosbree commented Oct 6, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

pcshrosbree commented Oct 5, 2026

Change

Notes

Validation (Linux x64)

Uh oh!

azure-pipelines Bot commented Oct 5, 2026

Uh oh!

dotnet-policy-service Bot commented Oct 5, 2026

Uh oh!

jkotas commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pcshrosbree commented Oct 5, 2026

Uh oh!

jkotas commented Oct 5, 2026

Uh oh!

jkoritzinsky commented Oct 6, 2026

Uh oh!

pcshrosbree commented Oct 6, 2026

Uh oh!

am11 commented Oct 6, 2026

Uh oh!

pcshrosbree commented Oct 6, 2026

Uh oh!

pcshrosbree commented Oct 6, 2026

Uh oh!

pcshrosbree commented Oct 6, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jkotas commented Oct 5, 2026 •

edited

Loading