Derive ilasm -DET identities from the assembly's content

[pcshrosbree]

With -DET, ilasm derives the MVID, the PE timestamp and the PDB ID from SHA-256 of the metadata block before any metadata has been written into it, so they depend only on the metadata's size. Different assemblies whose metadata is the same size share all three (#135214).

Depends on #135212. Only the last commit is new; I'll rebase onto main once #135212 merges.

Change

• PDB ID. Under -DET it is taken from the PdbChecksum, which Fix ilasm PdbChecksum and the PDB ID under -DET #135212 computes over the serialized PDB with its ID zeroed. The PDB is final apart from its ID at that point. PE-COFF.md describes this scheme, where one hash gives both the checksum and the ID. The CodeView entry carries this ID.
• MVID and PE timestamp. Both stay zero while the image is built, so the random MVID from DefineScope and the current time no longer reach the output.
  • At the end of CreatePEFile, a new ICeeFileGen::ComputeImageHash fixes up the image and hashes it as GenerateCeeFile will write it, through PEWriter::write(void**). ilasm supplies the hash function, since mscorpe has none. The hash covers the headers, the IL, the metadata, the resources and the debug directory.
  • The MVID and the timestamp are taken from that hash. The MVID is written into the already serialized metadata, at the Module row's index in the #GUID stream.
• The GUID and the stamp are still the first 16 and the next 4 bytes of the hash, as before.
• The in-memory writer was unreachable through the existing ICeeFileGen::GenerateCeeFile call path. It now sizes the image the way write(fileName) sizes the file. With /STRIPRELOC, that includes the stripped .reloc section's data, which write(fileName) still writes; before, the in-memory writer would have written past the end of its buffer.
• A second -DET defect: the export directory's timestamp was the current time, so -DET output with exports was not repeatable. It is now 0, which also keeps it out of the MVID.
• Tests:
  • TestDeterministicIdentity assembles pairs of inputs, with and without -DEBUG, from the same source path to the same output path. The pairs differ only in a method body, in metadata of the same size, or in sequence points. It checks that:
    • the MVID and the timestamp differ whenever the image differs;
    • the PDB ID differs exactly when the PDB's content does;
    • the same input gives the same bytes;
    • each identity is the start of the SHA-256 of its file with the identity fields zeroed.
  • TestDeterministicImageLayout checks repeatability and that hash rule for /STRIPRELOC, PE32+ and an assembly with an export. It also checks that the export timestamp is 0.
  • The test helper deletes earlier outputs before each run, so a stale file cannot pass for a new one.

Notes

• Where -DET output changes. Only the identity fields and the export timestamp.
  • The same IL was assembled in one directory with this branch and with Fix ilasm PdbChecksum and the PDB ID under -DET #135212. With -DEBUG, the DLLs differ in the COFF stamp, the MVID, the CodeView GUID, and the timestamps of the CodeView and PdbChecksum entries. The PDBs differ only in their ID. Without -DEBUG, only the COFF stamp and the MVID differ.
  • Output without -DET is unchanged: every change in ilasm is under m_fDeterministic, and the in-memory writer was not reached before.
• The MVID no longer equals the PDB GUID. The PE hash covers the CodeView entry, so the PDB ID has to be known first. System.Reflection.Metadata's PortablePdbBuilder and PEBuilder also use two hashes in that order.
• The hash is taken after fixup, not before. Hashing the section contents before relocation would miss inputs that reach only the headers. For example, /SUBSYSTEM and /BASE change only the optional header, and with this change they give different MVIDs. Fixing up early is safe because generateImage skips fixup() once it has run, and nothing between CreatePEFile and GenerateCeeFile changes the image apart from the MVID and the timestamp.
• Paths.
  • With -DEBUG: the MVID covers the absolute PDB path recorded in the CodeView entry. The IL round-trip determinism checks in CLRTest.Jit.targets assemble to the same path twice, so they are unaffected.
  • Without debug information: changing only the output directory leaves the tested image unchanged. The output file name can still reach the image, through the module name when .module is absent, and through the export directory.
  • The PDB ID depends on the PDB's content, which includes the IL source path as a document.
• Checked and Release ilasm give different identities for the same input. The Checked build's strncpy_s and strcpy_s fill the bytes after the terminator: with 0xFD on Linux (the PAL's), and with 0xFE on Windows (the MSVC debug CRT's). The filled bytes are after section-header names such as .text, .rsrc and .reloc, and after the import DLL name; Release writes 0x00 there. Those bytes are in the file, so the hash covers them.
  • This is pre-existing written content, not uninitialized memory, and it does not affect the shipped Release ilasm. Repeat runs are byte-identical within each configuration.
  • I can zero those bytes in a separate change.
• ICeeFileGen gains a virtual method at the end. mscorpe is a static library linked only into ilasm, which is its only consumer.
• This retains native ilasm's raw hash-byte mapping. BlobContentId.FromHash also sets the GUID's version and variant bits and the timestamp's high bit; I can switch to it if you prefer.

Validation (Linux x64 and Windows x64)

AI agents ran the builds, tests, mutation checks and probes on my machines; the results are summarized below.

• ilasm_tests against a Checked Core_Root, with counts from ilasm_tests.testResults.xml: 15 of 24 pass with Fix ilasm PdbChecksum and the PDB ID under -DET #135212 alone, where the 9 new cases fail, and 24 of 24 pass with this PR. Each of nine targeted mutants failed at least one new test case.
• The reproduction from ilasm: -DET derives the MVID, PE timestamp and PDB ID from a zero-filled buffer, so different assemblies share them #135214, with both the Checked and the Release ilasm:
  • the five variants get five distinct MVIDs, timestamps and PDB IDs;
  • repeat runs are byte-identical;
  • TryOpenAssociatedPortablePdb succeeds and the PdbChecksum matches the PE-COFF.md rule.
• Other switches. Repeat runs are also byte-identical for /DEBUG=OPT, /DEBUG=IMPL, /PDB, an EXE, /PE64 /X64, /STRIPRELOC, and an assembly with an export, with the two runs about 2 s apart. With Fix ilasm PdbChecksum and the PDB ID under -DET #135212 alone, the export case was not repeatable: its export timestamps were 1791211545 and 1791211547.
• IL round-trip. The generated IL round-trip determinism script (*_ilasmroundtrip.py) passes over the ilasm_tests output directory plus System.Collections, System.Linq and System.Private.CoreLib.
• Windows x64 (Windows Server 2025, MSVC 14.44), with ilasm built Checked at Fix ilasm PdbChecksum and the PDB ID under -DET #135212 and at this PR, and Release at this PR. ilasm_tests against a Checked Core_Root gives the same counts from the XML as on Linux: 15 of 24 with Fix ilasm PdbChecksum and the PDB ID under -DET #135212 alone and 24 of 24 with this PR. With both the Checked and the Release ilasm, the ilasm: -DET derives the MVID, PE timestamp and PDB ID from a zero-filled buffer, so different assemblies share them #135214 reproduction and the other switches above give the same results as on Linux, and each output's MVID, timestamp and PDB ID follow the hash rule the tests check. A Win32 resource (an rc.exe VERSIONINFO, converted with cvtres.exe, the form -RESOURCE reads) reaches the hash: changing one byte of it changes the MVID, which it did not with Fix ilasm PdbChecksum and the PDB ID under -DET #135212 alone. The IL round-trip and the mutants were not rerun on Windows.

Not run locally: macOS, an Arm64 host or a big-endian target, and the full src/tests suite. Arm64 and ARM output targets were tested from Linux x64. I can also run it on Windows ARM64 and Linux ARM64 hosts if that would help.

Resolves #135214

Note

This change, its validation and this description were prepared with AI assistance (Anthropic Claude and OpenAI Codex) under my direction. AI agents ran the builds, tests and probes on my machines. I reviewed the diff and this text before posting.

[azure-pipelines]

Azure Pipelines:
Successfully started running 4 pipeline(s).
12 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @dotnet/jit-contrib
See info in area-owners.md if you want to be subscribed.