Emit interop source directly as text - #133450
Merged
Merged
Conversation
Replace syntax-tree output with shared text emission and value-based signatures across native imports, COM/vtable, and JavaScript generators. Preserve marshalling stage ordering and incorporate review feedback and regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Move declaration extraction into input-analysis helpers and replace SyntaxKind with ContainingDeclarationKind in the containment model. Update native, COM, and JavaScript callers and cover declaration kinds, escaped names, nested modifiers, and pure-value emission. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Azure Pipelines: Successfully started running 4 pipeline(s). 12 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
|
Tagging subscribers to this area: @dotnet/interop-contrib |
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
JSMarshallingInfo now throws from GetHashCode despite being used as a value model for incremental caching, which is a correctness risk if hashed/combined anywhere.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (1)
| Severity | Finding |
|---|---|
 |
src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/JSMarshallingInfo.cs — JSMarshallingInfo overrides Equals but overrides GetHashCode to throw. This breaks the normal… |
What changed in this PR
This PR refactors the interop source generators to emit generated C# directly as deterministic text (instead of building/normalizing Roslyn syntax trees and round-tripping back to text), with shared writer/signature models and expanded incremental-generation verification.
Changes:
- Introduces shared text emission infrastructure (
IndentedTextWriter,CodeWriterHelpers,GeneratedMethodSignature/GeneratedParameter) and migrates core stub/marshalling emitters to write text per stage. - Updates COM/LibraryImport/JS generators and related marshallers to the writer-based pipeline and adjusts containment/signature modeling for incremental caching.
- Expands unit tests and documentation to validate caching behavior, deterministic formatting, and escaped identifier/literal handling.
| File | Description |
|---|---|
| src/libraries/System.Runtime.InteropServices/tests/LibraryImportGenerator.UnitTests/IncrementalGenerationTests.cs | Adds incremental-generation assertions for unchanged output and CRLF string outputs. |
| src/libraries/System.Runtime.InteropServices/tests/ComInterfaceGenerator.Unit.Tests/VTableGeneratorOutputShape.cs | Adds incremental caching test for generated text vs signature changes. |
| src/libraries/System.Runtime.InteropServices/tests/ComInterfaceGenerator.Unit.Tests/VerifyCompilationTest.cs | Adds shared incremental output verification helper and formatting assertions. |
| src/libraries/System.Runtime.InteropServices/tests/ComInterfaceGenerator.Unit.Tests/ComInterfaceGeneratorOutputShape.cs | Expands coverage for GUID bytes, escaped identifiers, and caching-by-signature. |
| src/libraries/System.Runtime.InteropServices/tests/ComInterfaceGenerator.Unit.Tests/ComClassGeneratorOutputShape.cs | Adds escaped identifier/generic nesting test and caching check for trivia-only edits. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/VariableDeclarations.cs | Switches variable/initialization emission from syntax nodes to text writer output. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Utils/SyntaxFactoryExtensions.cs | Removes syntax-factory helper utilities no longer used by text emission. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/UnmanagedToManagedStubGenerator.cs | Converts stub body generation to text emission with writer-composed try/catch/finally. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/TypeSymbolExtensions.cs | Removes type-to-syntax conversion helper (syntax tree no longer required for output). |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/TypePositionInfo.cs | Uses shared identifier escaping for parameter identifiers. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/TypeNames.cs | Replaces syntax-cached name/type syntax helpers with string-based helpers (e.g., CallConv). |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/SyntaxKindExtensions.cs | Maps Roslyn declaration kinds to a generator-owned declaration-kind enum. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/SyntaxExtensions.cs | Extracts trivia-insensitive containing namespace/type templates for text emission. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/SyntaxEquivalentNode.cs | Removes syntax-equivalence wrapper no longer needed with value/text models. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/SyntaxEquivalentComparer.cs | Removes syntax-equivalence comparer no longer used in output pipeline. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/SignatureContext.cs | Switches signature model to string/GeneratedParameter-based representation. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Microsoft.Interop.SourceGeneration.csproj | Links interpolated string handler attribute sources needed by writer handler usage. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/MarshallingAttributeInfo.cs | Simplifies linear collection marshalling info now that specialization is template-driven. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/StaticPinnableManagedValueMarshaller.cs | Emits pinning constructs as text and adapts to writer-based generator interface. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/MarshallingGeneratorExtensions.cs | Converts parameter/argument/return-type representation to string models. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/MarshallingGenerator.cs | Changes generator contract to write statements into a shared IndentedTextWriter. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/MarshallerHelpers.cs | Converts helper emitters/modifier logic from syntax nodes/tokens to strings. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/IidParameterIndexMarshallerResolver.cs | Rewrites QueryInterface marshalling stage emission as writer-driven text. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/IForwardedMarshallingInfo.cs | Changes forwarded-attribute API from syntax nodes to string attribute bodies. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/ICustomTypeMarshallingStrategy.cs | Updates strategy interface to emit via writer. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/Forwarder.cs | Updates forwarder marshaller to writer-based API (no emitted statements). |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/DelegateMarshaller.cs | Converts delegate marshalling emission to direct text. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/CustomTypeMarshallingGenerator.cs | Adapts custom marshalling generator to writer-based stage emission. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/CharMarshaller.cs | Updates char marshalling (including fixed header/body split) to text emission. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/BoolMarshaller.cs | Converts bool marshalling emit to writer-based text. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/BlittableMarshaller.cs | Emits blittable byref pin/marshal/unmarshal stages via text. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Marshalling/AttributedMarshallingModelGeneratorResolver.cs | Reworks native collection specialization to template-based string substitution. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/MarshalAsParser.cs | Converts MarshalAs forwarding attribute emission to string-based formatting. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/ManualTypeMarshallingHelper.cs | Adds type-name templating for placeholder replacement during collection specialization. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/ManagedTypeInfo.cs | Removes cached TypeSyntax creation from type info model. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/IndentedTextWriterExtensions.cs | Removes Roslyn-trivia-based writer helper (not needed with direct text emission). |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/IndentedTextWriter.cs | Adds shared writer with deterministic indentation and CRLF normalization. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/IncrementalValuesProviderExtensions.cs | Removes NormalizeWhitespace-based helper now that generation is string-based. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/IncrementalGeneratorInitializationContextExtensions.cs | Updates concatenated output registration to accept strings directly. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/GeneratedMethodSignature.cs | Adds value-based method/parameter signature model (including function-pointer formatting). |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/CustomMarshallingInfoHelper.cs | Preserves unmanaged slot type parameter when specializing linear collection marshallers. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/CodeWriterHelpers.cs | Adds shared helpers for escaping identifiers, string literals, and modifier ordering. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/BoundGenerators.cs | Switches target signature generation to GeneratedMethodSignature. |
| src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/ArrayMarshallingInfoProvider.cs | Aligns array marshalling info with updated linear collection info shape. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/Marshallers/StructAsHResultMarshallerFactory.cs | Ports marshaller emission to text writer. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/Marshallers/ObjectUnwrapperResolver.cs | Switches unwrapper marshalling info and emission to string-based modeling. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/Marshallers/ManagedHResultExceptionGeneratorResolver.cs | Converts HRESULT/exception marshalling emission and blob formatting to text. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/Marshallers/KeepAliveThisMarshaller.cs | Emits GC.KeepAlive(this) via text writer. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/Marshallers/ComInterfaceDispatchMarshallingResolver.cs | Emits dispatch marshalling via text writer. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/IncrementalMethodStubGenerationContext.cs | Converts calling convention model from syntax nodes to strings; adds ABI identifier helper. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/GeneratedStubCodeContext.cs | Replaces syntax-node stub representation with string/signature/attribute/body models. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/Comparers.cs | Removes syntax-equivalence-based comparers no longer needed. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/ComMethodContext.cs | Updates stub context construction for new stub representation. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/ComInterfaceInfo.cs | Switches to containing-syntax extraction helpers for text emission. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/ComInterfaceGeneratorHelpers.cs | Replaces embedded data blob syntax building with string formatting helper. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/ComClassInfo.cs | Switches to containing-syntax extraction helpers for text emission. |
| src/libraries/System.Runtime.InteropServices/gen/ComInterfaceGenerator/ComClassGenerator.cs | Changes COM class stub generation to IndentedTextWriter and tracks outputs as strings. |
| src/libraries/System.Runtime.InteropServices.JavaScript/tests/JSImportGenerator.UnitTest/Compiles.cs | Adds compilation and semantic assertions for escaped identifiers/literals and value-equality caching. |
| src/libraries/System.Runtime.InteropServices.JavaScript/tests/JSImportGenerator.UnitTest/CodeSnippets.cs | Adds new snippet sets for tasks, delegates, nesting, and escaped identifiers/literals. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/SignatureBindingHelpers.cs | Converts JS signature argument construction to string-based collection expression text. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/Marshaling/TaskJSGenerator.cs | Ports task marshalling emission to writer-based text. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/Marshaling/PrimitiveJSGenerator.cs | Ports primitive marshalling emission to writer-based text. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/Marshaling/ImplicitArgumentGenerator.cs | Ports implicit-argument setup emission to writer-based text. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/Marshaling/FuncJSGenerator.cs | Ports delegate marshalling emission to writer-based text with shared lambda formatter. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/Marshaling/BaseJSGenerator.cs | Updates base marshaller contract to writer-based emission and adds lambda formatter helper. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/JSSignatureContext.cs | Escapes method names and uses invariant formatting for generated identifiers. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/JSMarshallingInfo.cs | Adds value-based equality for JS marshalling info (including JSTypeArguments) for incremental caching. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/JSManagedTypeInfo.cs | Converts JS type info models to string-based types and adds value-based equality/hash where needed. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/JSGeneratorFactory.cs | Updates pattern matching for updated JS type info models. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/GeneratorDiagnostics.cs | Removes no-longer-needed syntax-type using. |
| src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/Comparers.cs | Removes syntax-equivalence-based comparers no longer needed. |
| docs/design/libraries/LibraryImportGenerator/Pipeline.md | Updates design doc to describe the new text emission pipeline and invariants. |
Extract JSON's containing-type traversal and declaration formatting into common utilities used by JSON and interop generators. Preserve JSON partial-type checks and symbol-formatted names, along with interop source spelling, raw attributes, and unsafe modifiers. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It is a broad cross-generator refactor of emission and incremental-caching behavior with a large blast radius that warrants thorough maintainer review despite strong test additions.
Review tier: Lite
Findings: 1
Pre-existing issues (1)
| Severity | Finding |
|---|---|
|
src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/JSMarshallingInfo.cs — JSMarshallingInfo overrides Equals but overrides GetHashCode to throw. This breaks the normal… View comment |
Replace custom record equality and hashing with SequenceEqualImmutableArray fields. Update parsing and resolution while preserving missing versus empty type arguments, and cover incremental invalidation when marshalling arguments change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The change is a large cross-cutting refactor of multiple source generators’ output pipelines, so a human review is needed to validate behavioral equivalence and edge cases across generators and TFMs.
Review tier: Lite
Findings: None
Issues resolved since last review (1)
| Severity | Finding |
|---|---|
|
src/libraries/System.Runtime.InteropServices.JavaScript/gen/JSImportGenerator/JSMarshallingInfo.cs — JSMarshallingInfo overrides Equals but overrides GetHashCode to throw. This breaks the normal… View resolved comment |
This was referenced Sep 9, 2026
Open
jtschuster
reviewed
Sep 10, 2026
jtschuster
reviewed
Sep 10, 2026
jtschuster
reviewed
Sep 10, 2026
jtschuster
approved these changes
Sep 11, 2026
jtschuster
left a comment
jtschuster
left a comment
Member
There was a problem hiding this comment.
I wasn't able to look closely at everything, but I checked the diff of the generated code for LibraryImportGenerator.Tests and ComInterfaceGenerator.Tests and it looks like it matches or improves on the existing generated code. I would personally prefer to have more gradual typed string wrappers where the syntax held in a string isn't clear (like IForwardedMarshallingInfo.TryCreateAttribute()), but this seems to work fine.
Preserve the direct-text generators while integrating unsafe-evolution support from main. Keep explicit unsafe bodies, conditional containing-type modifiers, shared declaration headers, and updated output tests. Enable the COM updated-rules verifier and cover callback signatures in JavaScript. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Call shared modifier and declaration helpers directly, remove unused type-name members, and document the bracket-free forwarded attribute contract. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This was referenced Sep 16, 2026
Avoid duplicating generic-parameter attributes on generated partial declarations. Include output-affecting COM declaration state in equality and hashing, and cover attribute resolution, modifier edits, and safety-mode changes with regression tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
jtschuster
approved these changes
Sep 22, 2026
Member
Author
|
/ba-g quic failure unrelated. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The interop generators currently build Roslyn syntax trees, normalize their whitespace, and then serialize them back to text for Roslyn to parse again. This replaces that output pipeline with direct text emission, reducing allocation and formatting overhead while keeping Roslyn for input analysis and code fixes.
Fixes #95882
Approach
StringBuilder-backedIndentedTextWriterand value-based parameter/signature models across LibraryImport, downlevel LibraryImport, COM/class/vtable, and JavaScript import/export generation.Validation
Built CLR + libraries with
build.cmd clr+libs -rc release, then rebuilt the affected generator projects after follow-up changes. Latest unit and functional runs, including outer-loop cases:JavaScript browser execution was not run.
Performance
Local measurements during development, before the final review cleanup and containment-model extraction, used Debug generators on a Release .NET 11 host (Windows x64, AMD EPYC 7763, Hyper-V). These are checkpoint measurements rather than a timing rerun of the final head.
Across 20 representative cold-generation workloads, 40 paired BenchmarkDotNet runs measured a 0.182 generation-time ratio and 0.323 allocated-bytes ratio versus the original implementation, using equal-weight geometric means. All 20 comparison pairs compiled without diagnostics and produced byte-identical assemblies; the escaped-entry-point fixes were verified separately.
The CoreLib build experiment did not establish an end-to-end build speedup. Six alternating pairs measured LibraryImport generation at 4.36 s -> 2.56 s, but complete project rebuilds averaged 110.45 s -> 115.10 s. The paired 95% interval for the whole-build change spanned -4.4 to +13.7 s, including zero.
complogconfirmed identical compiler arguments and captured inputs apart from the two interop generator DLLs.Note
This implementation and pull request description were prepared with GitHub Copilot.