Fix release Source Link mappings and validate package source coverage

BUILDGUIDE.md

@@ -282,6 +282,37 @@ Package Microsoft.Data.SqlClient.Extensions.Azure without building it beforehand
 dotnet build -t:PackAzure -p:PackBuild=false
 ```
 
+### Release Source Link Symbols
+
+To reproduce release symbol generation locally, set `BuildForRelease` in the environment before
+packing. `build.proj` launches child `dotnet` processes, so passing only
+`-p:BuildForRelease=true` to the orchestrator does not enable it in those processes.
+
+```powershell
+$env:BuildForRelease = 'true'
+dotnet build build.proj -t:PackSqlClient -p:Configuration=Release
+Remove-Item Env:\BuildForRelease
+```
+
+The `.nupkg` and matching `.snupkg` are written to
+`artifacts/Microsoft.Data.SqlClient/Project-Release/`. Inspect them together with
+[PackageValidator](tools/PackageValidator/README.md) or NuGet Package Explorer.
+
+To fail validation on missing source coverage or non-portable source paths:
+
+```powershell
+dotnet run --project tools\PackageValidator\src\PackageValidator.csproj -- `
+  artifacts\Microsoft.Data.SqlClient\Project-Release `
+  --fail-on missing-source-link --fail-on untracked-source --fail-on non-deterministic-source-path
+```
+
+Tracked source paths in the PDBs must match the Source Link document map; generated sources must
+be embedded. Keep the repository root configured in `RepositoryInfo.targets` as the source root:
+adding a separate `src/` root without source-control metadata can remap tracked files to `/_1/`
+while Source Link only maps `/_/`. NuGet Package Explorer reports this as
+**Contains untracked sources (obj)** for both Source Link and Deterministic, even when all `obj`
+sources are embedded and the compiler's deterministic flag is enabled.
+
 ## Versioning
 
 Versioning can be accomplished by using a mix of different parameters to the `build.proj` targets:

src/Directory.Build.props

@@ -141,7 +141,6 @@
           Condition="'$(SqlClientVersionsImported)' != 'true'" />
 
   <ItemGroup Condition="$(BuildForRelease) == 'true'">
-    <SourceRoot Include="$(MSBuildThisFileDirectory)/" />
     <EmbeddedFiles Include="$(GeneratedAssemblyInfoFile)" />
   </ItemGroup>
 

tools/PackageValidator/README.md

@@ -28,6 +28,8 @@ For every package it opens, PackageValidator reports and validates:
   judged only over implementation assemblies.
 - **Symbols** — sibling `.snupkg` matching by debug GUID with portable-PDB checksum verification,
   embedded-symbol detection, and orphan/mismatch reporting.
+- **Source coverage** — inspect embedded and matched portable PDBs for missing Source Link mappings,
+  unembedded generated/temporary sources, and non-normalized document paths, without fetching sources.
 - **Intrinsic rules** — severity-tagged findings for missing/mismatched symbols,
   unsigned/delay-signed assemblies, dependency inconsistencies, and more.
 - **Expected versions** — optional `--expect-*` assertions for inter-package version-match
@@ -80,6 +82,9 @@ or one of the following finding **categories**:
 - `symbol-checksum-mismatch`
 - `symbol-orphan`
 - `symbol-duplicate`
+- `missing-source-link`
+- `untracked-source`
+- `non-deterministic-source-path`
 - `delay-signed`
 - `unsigned`
 - `package-unsigned`
@@ -88,6 +93,34 @@ or one of the following finding **categories**:
 - `unexpected-file-version`
 - `unexpected-assembly-version`
 
+### Source coverage and deterministic paths
+
+The three source categories are warnings, and follow NuGet Package Explorer's offline checks:
+
+- `missing-source-link`: a document is neither embedded (Embedded Source custom debug information)
+  nor covered by a Source Link map. Matching is case-insensitive, supports exact keys and a single
+  trailing `*`, and does not download the resulting URL or verify remote source checksums.
+- `untracked-source`: an **unembedded** document has no Source Link mapping, or contains an `obj`,
+  `temp`, or `tmp` path segment (case-insensitive, either slash separator), even when Source Link maps it.
+- `non-deterministic-source-path`: a document path does not begin with `/_`, including embedded
+  documents. This checks path normalization, not complete build reproducibility.
+
+For example, `/_1/src/File.cs` is normalized but is **not** covered by a `/_/*` mapping.
+Generated documents under `/_/obj/` pass when embedded. Reference and satellite assemblies without
+PDBs do not receive source findings. `--no-snupkg` still checks embedded PDBs. Malformed source
+metadata/maps are inspection errors (exit 1), not successful coverage checks.
+Like NuGet Package Explorer, document records with nil name, language, hash algorithm, or hash
+handles are not inspected.
+
+JSON includes per-PDB `sourceCoverage` records on each inspected binary, with document counts and
+the offending paths; human-readable findings include the assembly, PDB, and document path.
+
+Gate these checks from the repository root (keep matching `.snupkg` files beside the packages):
+
+```powershell
+dotnet run --project .\tools\PackageValidator\src -- <package-directory> --fail-on missing-source-link untracked-source non-deterministic-source-path
+```
+
 ### `--expect-*` values
 
 Each `--expect-*` value is either `VALUE` (applies to every package in the run) or `id=VALUE`
@@ -219,6 +252,8 @@ The `test/` project is an xUnit v3 suite running on Microsoft.Testing.Platform.
 public-key-token computation, binary classification, SemVer 2.0 range evaluation (including
 prerelease ordering and malformed-input rejection), the rules engine, and expected-version
 assertions.
+Source coverage tests generate portable and embedded PDB fixtures with `MetadataBuilder`, exercising
+the extra-SourceRoot mapping regression, generated sources, path normalization, and symbol matching.
 
 ```bash
 # From tools/PackageValidator/test

tools/PackageValidator/src/AssemblyInspector.cs

@@ -85,7 +85,8 @@ public static BinaryReport Inspect(ZipArchiveEntry entry)
 
             // Read the debug directory to learn which PDB (by GUID) this assembly was built with,
             // whether it already carries an embedded portable PDB, and any recorded PDB checksums.
-            (Guid? codeViewGuid, bool hasEmbeddedPdb, List<PdbChecksum>? checksums) = ReadDebugInfo(pe);
+            (Guid? codeViewGuid, bool hasEmbeddedPdb, List<PdbChecksum>? checksums,
+                List<PdbSourceCoverage>? sourceCoverage) = ReadDebugInfo(pe);
 
             // File, informational, and target-framework versions are assembly-level custom
             // attributes, so scan for them by attribute type name.
@@ -132,6 +133,7 @@ public static BinaryReport Inspect(ZipArchiveEntry entry)
                 CodeViewGuid = codeViewGuid,
                 Checksums = checksums,
                 HasEmbeddedSymbols = hasEmbeddedPdb,
+                SourceCoverage = sourceCoverage,
             };
         }
         catch (BadImageFormatException)
@@ -167,12 +169,14 @@ private static SigningStatus DetermineSigningStatus(PEReader pe, bool hasPublicK
     /// embeds a portable PDB, and any recorded PDB checksums.
     /// </summary>
     /// <param name="pe">The PE reader positioned over the assembly.</param>
-    /// <returns>The CodeView GUID, the embedded-PDB flag, and the recorded checksums (if any).</returns>
-    private static (Guid? CodeViewGuid, bool HasEmbeddedPdb, List<PdbChecksum>? Checksums) ReadDebugInfo(PEReader pe)
+    /// <returns>The CodeView GUID, embedded-PDB flag, checksums, and embedded source coverage (if any).</returns>
+    private static (Guid? CodeViewGuid, bool HasEmbeddedPdb, List<PdbChecksum>? Checksums,
+        List<PdbSourceCoverage>? SourceCoverage) ReadDebugInfo(PEReader pe)
     {
         Guid? codeViewGuid = null;
         bool hasEmbeddedPdb = false;
         List<PdbChecksum>? checksums = null;
+        List<PdbSourceCoverage>? sourceCoverage = null;
 
         foreach (DebugDirectoryEntry entry in pe.ReadDebugDirectory())
         {
@@ -191,6 +195,15 @@ private static (Guid? CodeViewGuid, bool HasEmbeddedPdb, List<PdbChecksum>? Chec
 
                 case DebugDirectoryEntryType.EmbeddedPortablePdb:
                     hasEmbeddedPdb = true;
+                    try
+                    {
+                        using MetadataReaderProvider provider = pe.ReadEmbeddedPortablePdbDebugDirectoryData(entry);
+                        (sourceCoverage ??= []).Add(PortablePdb.ReadSourceCoverage(provider.GetMetadataReader(), "embedded"));
+                    }
+                    catch (BadImageFormatException ex)
+                    {
+                        throw new InvalidDataException("Invalid embedded portable PDB.", ex);
+                    }
                     break;
 
                 case DebugDirectoryEntryType.PdbChecksum:
@@ -211,7 +224,7 @@ private static (Guid? CodeViewGuid, bool HasEmbeddedPdb, List<PdbChecksum>? Chec
             }
         }
 
-        return (codeViewGuid, hasEmbeddedPdb, checksums);
+        return (codeViewGuid, hasEmbeddedPdb, checksums, sourceCoverage);
     }
 
     /// <summary>

tools/PackageValidator/src/Models.Report.cs

@@ -221,6 +221,9 @@ internal sealed class BinaryReport
     /// <summary>Gets the matching PDB's path within the symbol package, if any.</summary>
     public string? SymbolPackageFile { get; set; }
 
+    /// <summary>Gets source coverage for embedded and matched portable PDBs, or null when none were inspected.</summary>
+    public List<PdbSourceCoverage>? SourceCoverage { get; set; }
+
     /// <summary>
     /// Creates a report for a native or non-assembly DLL, recording its path, native version info,
     /// and marking it as unmanaged.
@@ -236,3 +239,22 @@ internal sealed class BinaryReport
         NativeVersion = nativeVersion,
     };
 }
+
+/// <summary>Offline source coverage and deterministic path checks for one portable PDB.</summary>
+internal sealed class PdbSourceCoverage
+{
+    /// <summary>Gets the symbol-package entry path, or <c>"embedded"</c> for an embedded portable PDB.</summary>
+    public required string Pdb { get; init; }
+
+    /// <summary>Gets the number of complete document records inspected, including embedded sources.</summary>
+    public required int DocumentCount { get; init; }
+
+    /// <summary>Gets documents neither embedded nor covered by a Source Link mapping.</summary>
+    public required List<string> MissingSourceLinkDocuments { get; init; }
+
+    /// <summary>Gets unembedded documents without a mapping or containing an obj, temp, or tmp segment even if mapped.</summary>
+    public required List<string> UntrackedDocuments { get; init; }
+
+    /// <summary>Gets document paths that do not begin with the deterministic <c>/_</c> prefix.</summary>
+    public required List<string> NonNormalizedDocuments { get; init; }
+}

tools/PackageValidator/src/PortablePdb.cs

@@ -3,7 +3,9 @@
 // See the LICENSE file in the project root for more information.
 
 using System.Reflection.Metadata;
+using System.Reflection.Metadata.Ecma335;
 using System.Security.Cryptography;
+using System.Text.Json;
 
 namespace PackageValidator;
 
@@ -19,6 +21,124 @@ internal static class PortablePdb
     /// <summary>The length in bytes of the PDB id (a 16-byte GUID followed by a 4-byte stamp).</summary>
     private const int PdbIdLength = 20;
 
+    private static readonly Guid SourceLinkKind = new("cc110556-a091-4d38-9fec-25ab9a351a6a");
+    private static readonly Guid EmbeddedSourceKind = new("0e8a571b-6926-466e-b4ad-8ab04611f5fe");
+
+    /// <summary>Inspects document coverage without downloading sources or decompressing embedded source content.</summary>
+    /// <param name="reader">The matched or embedded portable PDB metadata reader.</param>
+    /// <param name="pdb">The symbol-package entry path or <c>"embedded"</c>.</param>
+    /// <returns>The source documents that fail the offline Source Link or deterministic path checks.</returns>
+    /// <exception cref="InvalidDataException">The source metadata or Source Link map is malformed.</exception>
+    public static PdbSourceCoverage ReadSourceCoverage(MetadataReader reader, string pdb)
+    {
+        try
+        {
+            List<string> mappings = ReadSourceMappings(reader);
+            var missing = new List<string>();
+            var untracked = new List<string>();
+            var nonNormalized = new List<string>();
+            int documentCount = 0;
+            foreach (DocumentHandle handle in reader.Documents)
+            {
+                Document document = reader.GetDocument(handle);
+                if (document.Name.IsNil || document.Language.IsNil ||
+                    document.HashAlgorithm.IsNil || document.Hash.IsNil)
+                {
+                    continue;
+                }
+                documentCount++;
+                string path = reader.GetString(document.Name);
+                bool embedded = reader.GetCustomDebugInformation(handle)
+                    .Any(h => reader.GetGuid(reader.GetCustomDebugInformation(h).Kind) == EmbeddedSourceKind);
+                if (!embedded)
+                {
+                    bool mapped = !path.Contains('*') && mappings.Any(key => key.EndsWith('*')
+                        ? path.StartsWith(key[..^1], StringComparison.OrdinalIgnoreCase)
+                        : string.Equals(path, key, StringComparison.OrdinalIgnoreCase));
+                    if (!mapped)
+                    {
+                        missing.Add(path);
+                    }
+                    if (!mapped || path.Split(['/', '\\']).Any(segment =>
+                        segment.Equals("obj", StringComparison.OrdinalIgnoreCase) ||
+                        segment.Equals("temp", StringComparison.OrdinalIgnoreCase) ||
+                        segment.Equals("tmp", StringComparison.OrdinalIgnoreCase)))
+                    {
+                        untracked.Add(path);
+                    }
+                }
+                if (!path.StartsWith("/_", StringComparison.OrdinalIgnoreCase))
+                {
+                    nonNormalized.Add(path);
+                }
+            }
+            return new PdbSourceCoverage
+            {
+                Pdb = pdb,
+                DocumentCount = documentCount,
+                MissingSourceLinkDocuments = missing,
+                UntrackedDocuments = untracked,
+                NonNormalizedDocuments = nonNormalized,
+            };
+        }
+        catch (Exception ex) when (ex is BadImageFormatException or JsonException)
+        {
+            throw new InvalidDataException($"Invalid source metadata in PDB '{pdb}'.", ex);
+        }
+    }
+
+    /// <summary>Reads and validates module-level Source Link patterns using NPE's exact/trailing-wildcard rules.</summary>
+    /// <param name="reader">The portable PDB reader.</param>
+    /// <returns>Validated document path patterns; URL reachability is deliberately not checked.</returns>
+    private static List<string> ReadSourceMappings(MetadataReader reader)
+    {
+        var mappings = new List<string>();
+        foreach (CustomDebugInformationHandle handle in reader.GetCustomDebugInformation(
+            MetadataTokens.EntityHandle(TableIndex.Module, 1)))
+        {
+            CustomDebugInformation data = reader.GetCustomDebugInformation(handle);
+            if (reader.GetGuid(data.Kind) != SourceLinkKind)
+            {
+                continue;
+            }
+            using JsonDocument json = JsonDocument.Parse(
+                reader.GetBlobBytes(data.Value), new JsonDocumentOptions { AllowTrailingCommas = true });
+            if (json.RootElement.ValueKind != JsonValueKind.Object)
+            {
+                throw new InvalidDataException("Source Link must be a JSON object.");
+            }
+            foreach (JsonProperty property in json.RootElement.EnumerateObject())
+            {
+                if (property.Name != "documents")
+                {
+                    continue;
+                }
+                if (property.Value.ValueKind != JsonValueKind.Object)
+                {
+                    throw new InvalidDataException("Source Link 'documents' must be an object.");
+                }
+                foreach (JsonProperty mapping in property.Value.EnumerateObject())
+                {
+                    string key = mapping.Name;
+                    int star = key.IndexOf('*');
+                    if (key.Length == 0 || (star >= 0 && star != key.Length - 1) ||
+                        mapping.Value.ValueKind != JsonValueKind.String)
+                    {
+                        throw new InvalidDataException("Invalid Source Link document mapping.");
+                    }
+                    string url = mapping.Value.GetString()!;
+                    int urlStar = url.IndexOf('*');
+                    if (urlStar >= 0 && (star < 0 || urlStar != url.LastIndexOf('*')))
+                    {
+                        throw new InvalidDataException("Invalid Source Link URL wildcard.");
+                    }
+                    mappings.Add(key);
+                }
+            }
+        }
+        return mappings;
+    }
+
     /// <summary>
     /// Reads the debug GUID from a portable PDB.
     /// </summary>

tools/PackageValidator/src/SymbolResolver.cs

@@ -3,6 +3,7 @@
 // See the LICENSE file in the project root for more information.
 
 using System.IO.Compression;
+using System.Reflection.Metadata;
 
 namespace PackageValidator;
 
@@ -150,7 +151,7 @@ private static void MatchSymbolPackage(
                 asm.HasSymbolPackageSymbols = true;
                 asm.SymbolPackageSymbolsMatch = true;
                 asm.SymbolPackageFile = byGuid;
-                asm.SymbolPackageVerifiedByChecksum = VerifyMatchedChecksum(archive, byGuid, asm);
+                InspectMatchedPdb(archive, byGuid, asm);
                 matchedPdbs.Add(byGuid);
             }
             else if (pdbByPathKey.TryGetValue(StripExtension(asm.Path), out string? byPath))
@@ -201,18 +202,24 @@ private static void MatchSymbolPackage(
     }
 
     /// <summary>
-    /// Re-reads a GUID-matched PDB from the symbol package on demand and verifies it against the
-    /// assembly's recorded PDB checksums. Loading the bytes here (rather than retaining every PDB
-    /// during indexing) keeps peak memory to a single PDB.
+    /// Re-reads a GUID-matched PDB to verify checksums and inspect source coverage. Loading bytes
+    /// on demand rather than retaining all PDBs during indexing keeps peak memory to one PDB.
     /// </summary>
     /// <param name="archive">The open symbol-package archive.</param>
     /// <param name="pdbFullName">The full archive path of the matched PDB.</param>
     /// <param name="asm">The assembly whose checksums drive verification.</param>
-    /// <returns>The checksum verification result, or <see langword="null"/> if the entry cannot be read.</returns>
-    private static bool? VerifyMatchedChecksum(ZipArchive archive, string pdbFullName, BinaryReport asm)
+    private static void InspectMatchedPdb(ZipArchive archive, string pdbFullName, BinaryReport asm)
     {
         ZipArchiveEntry? entry = archive.GetEntry(pdbFullName);
-        return entry is null ? null : VerifyChecksum(asm, ReadEntry(entry));
+        if (entry is null)
+        {
+            return;
+        }
+        byte[] bytes = ReadEntry(entry);
+        asm.SymbolPackageVerifiedByChecksum = VerifyChecksum(asm, bytes);
+        using var stream = new MemoryStream(bytes, writable: false);
+        using MetadataReaderProvider provider = MetadataReaderProvider.FromPortablePdbStream(stream);
+        (asm.SourceCoverage ??= []).Add(PortablePdb.ReadSourceCoverage(provider.GetMetadataReader(), pdbFullName));
     }
 
     /// <summary>