Repository navigation
Add experimental document extraction over shared Documents abstractions - #7588
Merged
luisquintanilla merged 41 commits intoNov 5, 2026
Merged
Conversation
…ions.AI Introduces IOcrClient as a provider-neutral OCR/document-parsing capability in Microsoft.Extensions.AI.Abstractions, following the same abstraction + builder + middleware + DI shape as the existing capability family (IChatClient, ISpeechToTextClient, etc.). Abstractions (Microsoft.Extensions.AI.Abstractions): - IOcrClient, DelegatingOcrClient, OcrClientExtensions, OcrClientMetadata - OcrOptions, OcrResult, OcrPage, OcrTable, OcrTableCell, OcrBlock, OcrBoundingRegion, OcrUsage, OcrProgress Middleware + DI (Microsoft.Extensions.AI): - OcrClientBuilder, AsBuilder, AddOcrClient/AddKeyedOcrClient - LoggingOcrClient, OpenTelemetryOcrClient, ConfigureOptionsOcrClient and their builder extensions, mirroring the ISpeechToTextClient template All public surface is marked [Experimental] under the MEAI001 (AIOcr) diagnostic id. Includes unit tests for both libraries and updated API baselines.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new provider-neutral OCR / document-extraction capability to the Microsoft.Extensions.AI capability family, including abstractions, builder-based middleware pipeline, DI registration helpers, and OpenTelemetry/logging integrations.
Changes:
- Introduces
IOcrClient+ OCR result/options model types (pages, tables, blocks, bounding regions, usage/progress) inMicrosoft.Extensions.AI.Abstractions. - Adds
OcrClientBuilder+ DI registration extensions, plus middleware clients for logging, OpenTelemetry, and default-options configuration inMicrosoft.Extensions.AI. - Adds unit tests and updates API baseline JSONs and shared diagnostic IDs / OpenTelemetry constants to cover the new capability.
Reviewed changes
Copilot reviewed 40 out of 40 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/Libraries/Microsoft.Extensions.AI.Tests/Ocr/OpenTelemetryOcrClientTests.cs | Verifies OCR OpenTelemetry spans/tags and ActivitySource service exposure. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Ocr/OcrClientDependencyInjectionPatterns.cs | Validates DI registration and middleware wrapping patterns for OCR clients. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Ocr/OcrClientBuilderTests.cs | Tests builder pipeline ordering, null-handling, and service-provider flow-through. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Ocr/LoggingOcrClientTests.cs | Tests logging middleware behavior across log levels and DI-resolved logger factory. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Ocr/ConfigureOptionsOcrClientTests.cs | Tests options-cloning/configuration middleware behavior. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Microsoft.Extensions.AI.Tests.csproj | Shares TestOcrClient into the non-abstractions test project. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/TestOcrClient.cs | Adds an OCR test double for abstraction-level tests. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/OcrResultTests.cs | Tests OcrResult construction and markdown aggregation behavior. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/OcrOptionsTests.cs | Tests cloning semantics for OcrOptions. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/OcrClientMetadataTests.cs | Tests metadata property round-tripping and nullability behavior. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/OcrClientExtensionsTests.cs | Tests DataContent overload and argument validation in extensions. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/OcrBoundingRegionTests.cs | Tests bounding-region helpers and bounds calculation. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/DelegatingOcrClientTests.cs | Tests delegating client pass-through semantics and GetService behavior. |
| src/Shared/DiagnosticIds/DiagnosticIds.cs | Adds a new experimental diagnostic ID bucket for OCR (AIOcr). |
| src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs | Adds a non-standard GenAI usage attribute for pages processed. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/OpenTelemetryOcrClientBuilderExtensions.cs | Adds .UseOpenTelemetry(...) builder extension for OCR pipelines. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/OpenTelemetryOcrClient.cs | Implements OpenTelemetry instrumentation for OCR operations (spans + metrics). |
| src/Libraries/Microsoft.Extensions.AI/Ocr/OcrClientBuilderServiceCollectionExtensions.cs | Adds AddOcrClient / AddKeyedOcrClient DI registration helpers. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/OcrClientBuilderOcrClientExtensions.cs | Adds IOcrClient.AsBuilder() convenience extension. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/OcrClientBuilder.cs | Adds the OCR pipeline builder implementation (Use(...), Build(...)). |
| src/Libraries/Microsoft.Extensions.AI/Ocr/LoggingOcrClientBuilderExtensions.cs | Adds .UseLogging(...) builder extension for OCR pipelines. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/LoggingOcrClient.cs | Adds logging middleware for OCR operations with sensitive-data gating. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/ConfigureOptionsOcrClientBuilderExtensions.cs | Adds .ConfigureOptions(...) builder extension for defaulting/cloning options. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/ConfigureOptionsOcrClient.cs | Implements options-configuration middleware for OCR operations. |
| src/Libraries/Microsoft.Extensions.AI/Microsoft.Extensions.AI.json | Updates API baseline to include OCR middleware/builder/DI additions. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Utilities/AIJsonUtilities.Defaults.cs | Registers OCR types for source-generated JSON serialization defaults. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrUsage.cs | Adds OCR usage model (pages processed + additional properties). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrTableCell.cs | Adds structured table-cell model (indices, spans, content, kind). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrTable.cs | Adds structured table model (cells or markdown + optional geometry). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrResult.cs | Adds OCR result model (pages + markdown aggregation + usage/raw/additional). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrProgress.cs | Adds progress model for long-running OCR operations. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrPage.cs | Adds per-page OCR model (markdown + tables/blocks/confidence). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrOptions.cs | Adds request options model (model id, include images, additional properties). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrClientMetadata.cs | Adds client metadata model (provider name/uri/default model). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrClientExtensions.cs | Adds GetService<T> + DataContent overload for OCR invocation. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrBoundingRegion.cs | Adds polygon-based geometry primitive + rectangle helper + bounds computation. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrBlock.cs | Adds layout-block model (text/kind/geometry/confidence). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/IOcrClient.cs | Introduces OCR client abstraction (unary call + IProgress + GetService). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/DelegatingOcrClient.cs | Adds delegating base class for OCR pipeline middleware. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Microsoft.Extensions.AI.Abstractions.json | Updates API baseline to include new OCR abstraction surface. |
Adds an optional OcrImage sink so engines can return page images/figures when OcrOptions.IncludeImages is requested. Every member is optional: document-native engines (Mistral OCR inline images, Azure Document Intelligence figures) populate Content with rendered bytes; a vision-LLM transcriber may populate only Caption. One shape serves both provider archetypes.
Symmetric with the existing DataContent overload. Handles self-contained data: URIs by delegating to the DataContent path and does no file/network IO. For file: and remote URIs it throws NotSupportedException, leaving native URL passthrough (Mistral document_url, Azure DI uriSource) as an explicit open design question.
The method returns a rich OcrResult (markdown, tables, figures/images, blocks, confidence, language), not plain text, so GetTextAsync was a misnomer. ExtractAsync names the document-extraction operation accurately. Scoped to the Ocr/ types only; ISpeechToTextClient.GetTextAsync is unrelated and unchanged. Alternatives considered for maintainer discussion: GetDocumentAsync, AnalyzeAsync.
…ines Complete the source rename across the OCR test suites in both Microsoft.Extensions.AI.Abstractions.Tests and Microsoft.Extensions.AI.Tests, and regenerate the two API baselines (which also reflect the round-2 members ExtractFromUriAsync, OcrImage, and OcrPage.Images). Also fix pre-existing StyleCop/Sonar violations in the OCR tests that were blocking the core test project from compiling.
Stream.CopyToAsync(Stream, CancellationToken) does not exist on net462/ netstandard2.0, so the 2-arg call failed to compile (CS1503) on the net462 leg. Use the #if !NET conditional buffer-size overload, matching the existing DataContent.cs convention.
Introduce OcrPoint and OcrBoundingBox readonly record structs and retype OcrBoundingRegion.Polygon as IReadOnlyList<OcrPoint> with a typed OcrBoundingBox GetBounds() return. This makes odd or empty vertex lists unrepresentable (fixing the GetBounds inverted-bounds edge case) and reads self-documenting, while still carrying a rotation-skewed quadrilateral. Rename OcrPage.Index (zero-based) to OcrPage.PageNumber (one-based) so it correlates directly with OcrBoundingRegion.PageNumber, matching Azure Document Intelligence, PdfPig, and human page conventions, and removing the off-by-one adjustments providers had to make. Update the internal OcrDocumentReader and the OCR tests, and regenerate the Microsoft.Extensions.AI.Abstractions API baseline.
Family-consistency changes surfaced by an API-review rehearsal against the sibling abstractions (IChatClient, ISpeechToTextClient): - Streaming: add IAsyncEnumerable<OcrResponseUpdate> via ExtractStreamingAsync (the unary/streaming/GetService triad, matching IChatClient) plus a ToOcrResult/ToOcrResultAsync reducer; retire IProgress<OcrProgress> and the OcrProgress type (progress folds onto OcrResponseUpdate). - Unseal the OCR DTOs (OcrOptions, OcrResult, OcrPage, OcrBlock, OcrTable, OcrTableCell, OcrImage, OcrBoundingRegion, OcrUsage) to mirror the unsealed ChatResponse/ChatOptions family shape. - Geometry units: add OcrPage.Width/Height and an OcrCoordinateUnit (ChatRole-style open struct: Pixel/Inch/Normalized) so bounding coordinates are interpretable across engines. - Model OcrBlock.Kind and OcrTableCell.Kind as ChatRole-style open structs (OcrBlockKind, OcrTableCellKind) instead of raw strings. - OcrBoundingRegion.GetBounds() now returns OcrBoundingBox? (null on empty polygon) instead of an ambiguous all-zero default. - Remove the leaky OcrResult.OcrSource; document OcrTable.Cells as authoritative when non-null. - Add <exception> docs to the AddOcrClient/AddKeyedOcrClient overloads. Updates the middleware (Logging/OpenTelemetry/ConfigureOptions), the MEDI OcrDocumentReader consumer, OCR tests, and both API baselines accordingly.
…es, usage tokens Continue the IOcrClient reshape ahead of API review (dotnet#7587/dotnet#7588). - Replace parallel OcrPage.Blocks/Tables/Images with a single reading-order Elements list over a new polymorphic OcrElement base (OcrBlock/OcrTable/OcrImage derive); consumers project with OfType<T>(). - Add optional nested OcrTableCell.Elements for structured cell content; keep the Content string as a flat-text convenience. - Convert OcrCoordinateUnit to a closed enum and add Point; add an OcrCoordinateOrigin enum; move both to document level (OcrResult/OcrPageResult) and remove the per-page CoordinateUnit. - Keep OcrBlockKind and OcrTableCellKind as open structs; add RowHeader and RowSection well-known cell kinds. - Add nullable Input/Output/TotalTokenCount to OcrUsage for the vision-LLM path. - Rename Markdown to Text on OcrPage/OcrResult; drop response ModelId, OcrPage.Confidence, and OcrOptions.IncludeImages; FromRectangle takes float. - Keep ToOcrResult/ToOcrResultAsync reducers and the convenience overloads as extension methods, matching the MEAI ChatResponseExtensions precedent. - Regenerate the ApiChief baselines for the reshaped surface. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
SPIKE-07/08 primary-source verification refuted the round-1 assumption that a
document uses one uniform coordinate unit. Google Document AI models
Page.Dimension{width,height,unit} per page, and Azure DI's per-page
DocumentPage.Unit is "pixel" for image inputs and "inch" for PDF, so one
analyze call over a mixed batch returns different units on different pages.
- Group OcrPage.Width/Height into a new OcrPageDimensions readonly record
struct (the extent; an atomic pair several engines bundle).
- Move CoordinateUnit + CoordinateOrigin from the document level (OcrResult,
OcrPageResult) onto OcrPage; they are siblings to the dimensions extent,
together describing the per-page coordinate system.
- Drop the reducer's last-non-null coordinate carry-forward: pages already
carry their own unit/origin.
- Defer continuous page-rotation angle (Azure-family-only) to
AdditionalProperties for v1; element rotation is already carried losslessly
by the polygon OcrBoundingRegion.
- Update per-page tests and regenerate the ApiChief baseline (32 public types).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
SPIKE-06 built an exhaustive 14-engine raw-output inventory to justify the RawRepresentation/AdditionalProperties escape hatches, then a promote interview resolved which recurring un-homed fields become first-class. - Add BoundingRegion, Confidence, RawRepresentation, and AdditionalProperties to OcrTableCell, mirrored member-for-member from OcrElement. Five engines (Textract, Google Document AI, Azure DI, Adobe, Docling) emit per-cell geometry, and a cell has its own rectangle even when empty or padded. OcrTableCell deliberately does not derive from OcrElement (a cell is not a page-reading-order element) and no shared base type is extracted yet; mirroring the members keeps that a reversible way-station under [Experimental]. - Add [JsonIgnore] object? RawRepresentation to OcrPage. Every other node carries it; the page node was the lone exception. Because the page rides through ToOcrResult reduction, provider-native page data now survives into OcrResult.Pages, closing a silent per-page raw data-loss. - Detected language is deferred (not promoted) to a standardized AdditionalProperties key rather than a lossy typed field. - Add OCR tests for cell geometry round-trip and per-page raw survival; regenerate the ApiChief baseline (five additive members, no new types). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
Move the reshaped OCR surface out of Microsoft.Extensions.AI(.Abstractions) into a standalone peer library, Microsoft.Extensions.DocumentExtraction, mirroring VectorData / DataIngestion. OCR is document extraction rather than an AI-branded concern, so it gets its own package + namespace (Microsoft.Extensions.DocumentExtraction) with domain-noun naming (Document*/DocumentExtraction*) instead of the MEAI-internal verb scheme. - New Microsoft.Extensions.DocumentExtraction.Abstractions: the Document* model, IDocumentExtractionClient, DocumentExtractionResult/Options/ ClientMetadata/PageResult, extensions, DelegatingDocumentExtractionClient. - New Microsoft.Extensions.DocumentExtraction: builder + Logging/ OpenTelemetry/ConfigureOptions delegating clients + DI/builder extensions, with self-contained internal telemetry helpers (no MEAI-core coupling). - Remove the OCR types from Microsoft.Extensions.AI(.Abstractions) and shrink both API baselines accordingly (only OCR type-blocks removed; the manual params edits and JSONC comments are preserved byte-for-byte). - Populate the two new Stage=normal baselines (peer precedent: VectorData ships a populated baseline for its [Experimental] surface). - Repoint Microsoft.Extensions.DataIngestion's OcrDocumentReader onto DocumentExtraction.Abstractions. - Add DiagnosticIds MEDE0001 for the new [Experimental] surface. - Retarget all OCR tests to two new DocumentExtraction test projects. Green across all five TFMs (netstandard2.0, net462, net8.0, net9.0, net10.0); DocumentExtraction tests 31/31 + 29/29 on net8/9/10; the MEDI OcrDocumentReader test stays green after the repoint. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
luisquintanilla
changed the title
The OCR test helper moved to Microsoft.Extensions.DocumentExtraction.Abstractions.Tests (TestDocumentExtractionClient.cs) during the extraction, but Microsoft.Extensions.AI.Tests.csproj still linked the moved-away ..\Microsoft.Extensions.AI.Abstractions.Tests\TestOcrClient.cs, causing error CS2001 (source file not found) on all TFMs in CI. No MEAI test references any Ocr type, so the link is safe to delete. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
Every packable library in the repo ships a README.md (auto-included as PackageReadmeFile on pack); the two new DocumentExtraction packages were missing theirs, so packing failed with NU5019. Add READMEs mirroring the DataIngestion peers, describing the document-extraction capability and the Abstractions-vs-core package split. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
Open
6 tasks
Member
|
Just waiting to see if I can merge my Apple Intelligence implememntation... dotnet/maui-labs#450 No pressure, but I need more AI abstractions to implement! |
Verify typed source ordering and snapshots, metadata transfer and borrowing, reduced public API, native access independent of content capture, recursive policy enforcement and existing cleanup guarantees across supported test targets. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use explicit fixture ownership and test-context task creation while preserving primary-error, cancellation and borrowed-native lifetime assertions. Leave pre-existing file-size style warnings outside this bounded correction. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep canonical content and typed source ownership while removing unused element bags, extension envelopes and redundant authoring helpers. Use optional text/cell roles and explicit supplied Markdown. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Compile-prerequisite snapshot, not final acceptance: return Task<Document> directly, remove public staging and finalization protocol, snapshot delegation options and log lifecycle without serializing payloads. Preserve caller input and returned document ownership, including cleanup when completion logging fails. Regenerate real API baselines and document independent Documents/Extraction/consumer composition. Final direct-contract regression validation follows separately. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replace retired staging-only regressions with direct canonical-document exchange, source-body/native lifetime, options snapshot, cancellation/error and middleware composition tests. Verify unreturned documents are released when completion logging fails and preserve primary errors on cleanup or failure logging. Actual four-target runners and integrated build pass; staging retirements and quality review are recorded in isolated session artifacts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1650980&view=codecoverage-tab |
Preserve the reviewed data model, remove structural rendering, and correct client lifetime and cleanup contracts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1658890&view=codecoverage-tab |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1660146&view=codecoverage-tab |
tarekgh
reviewed
Nov 4, 2026
tarekgh
reviewed
Nov 4, 2026
tarekgh
reviewed
Nov 4, 2026
Migrate composition and DI to the abstract client, preserve extraction and cleanup semantics, qualify logging failures, and remove the unused logging configuration callback. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copy exactly six reviewed Documents source, API inventory, documentation and regression-test blobs from e08fe2e so Extraction provides one coherent public bootstrap source snapshot. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1664718&view=codecoverage-tab |
Incorporate c3bb497 without changing the Extraction contract. Retain the existing Extraction experimental diagnostic integration and remove obsolete prerequisite guidance. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1665043&view=codecoverage-tab |
Document package selection, installation and practical extraction, logging and DI examples without development history or PR lifecycle guidance. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
jeffhandley
approved these changes
Nov 5, 2026
luisquintanilla
enabled auto-merge (squash)
November 5, 2026 03:24
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1665173&view=codecoverage-tab |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a provider-neutral, whole-document extraction capability over the shared
Documents model, which is now in
mainthrough#7835.
The result is
ExtractionDocument, not an ingestion-specific type.Applications can use extraction without MEDI or a particular provider SDK.
What this PR adds
Microsoft.Extensions.DocumentExtraction.AbstractionsDocumentExtractionClient, options, service lookup andDelegatingDocumentExtractionClient.Microsoft.Extensions.DocumentExtractionDocumentExtractionClientreplaces the former interface without a compatibilityshim. This is a deliberate source/binary-breaking preview change: provider
implementations and consumers must migrate and rebuild together.
The existing operation remains
Task<ExtractionDocument> ExtractAsync(Stream, identifier, mediaType, options, cancellationToken). The abstract root suppliesvirtual service lookup and optional protected
Dispose(bool)cleanup.It adds no Pipes overload, page-streaming protocol, async disposal or global
disposed-state policy. Future operations still need useful defaults and
explicit middleware/ownership semantics.
Documents prerequisite is now merged
Merged
mainatc3bb4972d9f7d1fb112681f3d3d9436e7bb686f8into thisbranch without rebasing or rewriting the existing review history.
Current head:
2f2fc084360b9b46281bc368b4353c559525abb6.All 23 Documents production/test/API/documentation blobs match
mainexactlyand no longer appear in this PR's diff. They remain in the repository as the
merged shared library. The remaining 23 paths cover the two Extraction
libraries, their tests and required experimental-diagnostic integration.
The document hierarchy, mixed-content table cells, text/image primitives,
source pages and geometry belong to Documents, not a second result model here.
This PR does not change the merged Documents API.
Composition, errors and ownership
Middleware follows the MEAI decorator/builder composition pattern: the first
registered wrapper is outermost. The deliberate distinction is an abstract
primary client instead of MEAI's
IChatClientinterface.With application-defined provider and middleware factories:
MyExtractionMiddlewarederives fromDelegatingDocumentExtractionClient.Ordinary and keyed factory registrations support the configured DI lifetimes.
Supplied-instance registration remains singleton-only; the built pipeline owns
its configured clients. Factories must not share a disposable client between
independently owned pipelines.
Caller input streams and opaque native values remain borrowed. Per-call option
snapshots preserve present-null entries and borrowed opaque values without
mutating the caller's options. The document is data, not a disposal boundary.
Native inspection is not a promise of durable original wire bytes or lifetime
independence.
Lifecycle logging records events, not document bodies, native results,
exceptions or option values. The unused logging configuration callback is
removed. When logging succeeds, results and original failure/cancellation
propagate unchanged. Invocation logging can prevent a provider call; completion
logging can prevent delivery of an otherwise successful result. If failure or
cancellation logging also throws,
AggregateExceptionretains the originalprovider exception first and the logging exception second. A cancellation plus
logging failure therefore faults rather than preserving canceled task status.
Failed pipeline construction likewise preserves the construction error first
if cleanup also fails.
Separate downstream work
Provider implementations live in
CommunityToolkit/AI#82;
these libraries ship no extraction engine or provider SDK.
MEDI's optional reader bridge lives in
#7904. It wraps the exact
extracted graph in the retained
IngestionDocumentenvelope, keeping ingestionidentity separate. No MEDI dependency, chunking or vector-store behavior is
introduced here.
Validation and review
The merge retains the existing Extraction C#, tests and API inventories
unchanged. One conflict in
Directory.Build.targetsretained main'sexperimental-diagnostic configuration plus the existing
MEDE0001entry.The owning README removes obsolete prerequisite notes and reflects main's
additional .NET 11 target.
Using main's SDK
11.0.100-rc.3.26522.148, focused nonincremental owningbuilds succeed with zero warnings/errors. Production targets are net11,
net10, net9, net8, net462 and netstandard2.0.
All 295 focused executions passed, zero failures/skips. A first no-restore
attempt found old assets lacking net11; normal targeted restore/build resolved
that without changing dependency pins or suppressing warnings.
Real ApiChief output from the new net11 binaries is structurally identical
to both checked-in inventories apart from the assembly-name version changing
from 10.10 to 11.0. The tool needed an isolated net11 reference context with
byte-identical built assemblies; four existing tool-only warning categories
remain disclosed separately from the clean owning builds. No public API or
inventory edit was introduced by this merge.
Post-push hosted CI is running/queued; no successful hosted result for this
new head is claimed here.
Review the client/options/service-lookup contracts first, then delegation, builder
and DI lifetimes, then cancellation, construction cleanup and logging.
Related design discussion:
#7587.
Historical private candidate packages and earlier downstream/hosted results
remain separate, immutable evidence for their exact inputs. They do not certify
this new merge revision, and this update does not republish or rebind them.
Successful checks do not constitute Extraction API approval, an upstream merge
or a package release. No live/paid provider request is part of this update.
Microsoft Reviewers: Open in CodeFlow