Repository navigation
Promote model package API to stable - #33166
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Stable C API output contracts, Python GIL handling, and missing Python coverage must be addressed.
4 open findings
What changed in this PR
Promotes model-package functionality from the experimental lookup mechanism to a stable, versioned C/C++ API.
Changes:
- Adds stable C and C++ model-package APIs.
- Restores Python bindings and updates tests.
- Removes experimental declarations and refreshes documentation.
| File | Description |
|---|---|
onnxruntime/test/autoep/test_model_package.cc |
Migrates tests to the stable API. |
onnxruntime/python/onnxruntime_pybind_state.cc |
Restores Python bindings. |
onnxruntime/core/session/ort_apis.h |
Declares the API getter. |
onnxruntime/core/session/onnxruntime_c_api.cc |
Registers the stable API table. |
onnxruntime/core/session/model_package/README.md |
Documents stable API usage. |
onnxruntime/core/session/model_package_api.h |
Declares internal implementations. |
onnxruntime/core/session/model_package_api.cc |
Implements the companion API table. |
onnxruntime/core/session/model_compilation_options.h |
Removes an obsolete experimental include. |
onnxruntime/__init__.py |
Exports model-package Python types. |
include/onnxruntime/core/session/onnxruntime_experimental_c_api.inc |
Removes experimental entries. |
include/onnxruntime/core/session/onnxruntime_experimental_c_api.h |
Removes experimental opaque types. |
include/onnxruntime/core/session/onnxruntime_cxx_inline.h |
Implements C++ wrappers. |
include/onnxruntime/core/session/onnxruntime_cxx_api.h |
Declares C++ wrappers. |
include/onnxruntime/core/session/onnxruntime_c_api.h |
Defines the stable public C API. |
🧠 Review effort: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Xiaoyu Z (xiaoyu-work)
force-pushed
the
xiaoyu/promote-model-package-api
branch
from
October 7, 2026 22:06
0501673 to
c2f5bf9
Compare
Tianlei Wu (tianleiwu)
requested changes
Oct 7, 2026
Tianlei Wu (tianleiwu)
left a comment
Tianlei Wu (tianleiwu)
left a comment
Contributor
There was a problem hiding this comment.
The stable companion table is appended without reordering existing OrtApi members, and the experimental removal and documentation updates are consistent. I rechecked the earlier review threads: the C API fixes are present, and the Python concerns no longer apply to this scoped promotion.
There is one remaining stable failure-output contract violation, detailed inline, plus a recommendation to add coverage for the new C++ wrappers. A standalone probe compiled the current model_package_api.cc and model_package_context.cc against existing CPU support archives: it reproduced the failing count contract, while the existing-component happy path and C++ wrapper metadata/selection/session/RAII smoke passed. Standalone C and C++ public-header syntax checks also passed. A full runtime rebuild and the current-head gtest suites were not run.
Tianlei Wu (tianleiwu)
approved these changes
Oct 8, 2026
Tianlei Wu (tianleiwu)
enabled auto-merge (squash)
October 8, 2026 03:54
This was referenced Oct 9, 2026
Franklin Moormann (ooples)
pushed a commit
to ooples/AiDotNet
that referenced
this pull request
Oct 13, 2026
Updated [Microsoft.ML.OnnxRuntime](https://github.com/Microsoft/onnxruntime) from 1.30.0 to 1.31.0. <details> <summary>Release notes</summary> _Sourced from [Microsoft.ML.OnnxRuntime's releases](https://github.com/Microsoft/onnxruntime/releases)._ ## 1.31.0 ONNX Runtime 1.31.0 stabilizes model-package and EPContext data APIs, improves CPU model loading and quantized MoE inference, expands device-based execution-provider selection, and strengthens model-loading and runtime reliability. These notes cover changes since ONNX Runtime 1.30.0. CUDA and WebGPU kernel updates are summarized here; detailed provider notes are covered by their separate plugin EP releases. ## Highlights - Promoted the model-package API and EPContext data callbacks to stable C and C++ APIs, with EPContext callback support added across language bindings ([#33166](microsoft/onnxruntime#33166), [#32265](microsoft/onnxruntime#32265)). - Reduced CPU session-initialization overhead by parallelizing eligible weight prepacking, and accelerated block-wise INT4/INT8 QMoE experts with MLAS QNBit kernels and grouped expert dispatch ([#31691](microsoft/onnxruntime#31691), [#32644](microsoft/onnxruntime#32644), [#32668](microsoft/onnxruntime#32668)). - Added x86 FP16 LayerNorm/RMSNorm acceleration, Arm KleidiAI SVE2.1 FP16 GEMM support, and additional RISC-V vector kernels ([#32715](microsoft/onnxruntime#32715), [#32670](microsoft/onnxruntime#32670), [#32710](microsoft/onnxruntime#32710)). - Enabled CoreML participation in device-based EP selection, including Apple Neural Engine selection through `PREFER_NPU` ([#31975](microsoft/onnxruntime#31975)). - Added workspace-memory accounting and verification for constrained-memory graph partitioning, alongside broader model, graph, and tensor validation ([#31962](microsoft/onnxruntime#31962), [#32189](microsoft/onnxruntime#32189)). ## Announcements & Compatibility - [CUDA Plugin EP v0.3.0](https://github.com/microsoft/onnxruntime/releases/tag/plugin-ep-cuda/v0.3.0) and WebGPU Plugin EP v0.5.0 are matched with ONNX Runtime 1.31.0. For **contrib-op compatibility**, core and the plugin must use the same contributed-operator schemas; building both from the **same ONNX Runtime revision** is recommended. Schema mismatches can cause incorrect execution or crashes and may not be detected during plugin registration. - **`GatherBlockQuantized` behavior change:** out-of-range indices now produce zero output slices across CPU, CUDA, and WebGPU for both integer and floating-point quantized formats. This deliberately differs from ONNX `Gather` error semantics ([#32480](microsoft/onnxruntime#32480)). - **Experimental API migration:** model-package consumers should use `OrtApi::GetModelPackageApi` and the stable `OrtModelPackageApi` table. EPContext callback registration now uses stable APIs; the corresponding experimental lookup entries were removed. The stable native callback contract carries callback/state pointers, with payload limits enforced by application callbacks, bindings, or EPs rather than a native read-options policy ([#33166](microsoft/onnxruntime#33166), [#32265](microsoft/onnxruntime#32265)). - **ACL EP is deprecated** and will be removed in a future release. Source builds using `--use_acl` now receive a deprecation warning ([#32564](microsoft/onnxruntime#32564)). - The ONNX dependency remains at **1.22.0**. The initial ONNX 1.23 integration was reverted pending upstream fixes; this release does not ship that integration's new opset or FLOAT6 support ([#33073](microsoft/onnxruntime#33073)). - Supported native builds now default to **1DS telemetry**. Windows TraceLogging remains selectable with `--use_windows_telemetry` or `onnxruntime_USE_WINDOWS_TELEMETRY=ON`; telemetry can be disabled with `--no_telemetry` ([#32384](microsoft/onnxruntime#32384)). ## New Features ### Core APIs & Runtime - Stabilized the model-package C and C++ APIs for loading packages, selecting compatible model variants, and creating sessions ([#33166](microsoft/onnxruntime#33166)). - Stabilized named EPContext data read/write callbacks for application-managed storage. Added EP support negotiation so registered callbacks do not silently fall back to filesystem I/O ([#32265](microsoft/onnxruntime#32265)). - Extended the Compile API to write external initializers and EP context data to buffers, enabling in-memory compilation workflows for models larger than 2 GB ([#31347](microsoft/onnxruntime#31347)). - Added workspace-memory accounting, source reporting, and post-partition reservation verification. Set `session.strict_workspace_verification=1` to fail initialization on workspace declaration overruns or nonzero orphaned reservations; strict verification is not supported for ORT-format model loads ([#31962](microsoft/onnxruntime#31962), [#32189](microsoft/onnxruntime#32189)). - Added the opt-in source-build option `onnxruntime_DISABLE_DEVICE_DISCOVERY=ON` to skip platform accelerator probing and report only the CPU device ([#32592](microsoft/onnxruntime#32592)). ### Shared Contrib Operators These updates affect core operator schemas and cross-provider behavior, not only the separately released GPU plugins. - Extended `EngramGate` and `NGramHashMapping` for Qwen4-Exp, and added `VarlenNGramHashMapping` for packed DeepSeek Engram batching with request-local history. Packed hashing also supports Qwen4-Exp head offsets, EOS resets, and segment boundaries ([#32285](microsoft/onnxruntime#32285), [#32358](microsoft/onnxruntime#32358), [#32467](microsoft/onnxruntime#32467)). - Added the `activation` attribute to `GatedRMSNorm` for SiLU, its Swish alias, and Sigmoid gating across CPU, CUDA, and WebGPU. SiLU remains the default ([#32512](microsoft/onnxruntime#32512)). - Added weightless hyper-connection operators `BranchwiseRMSNorm`, `ScaledSiLU`, `HyperConnectionPreMix`, and `HyperConnectionPostMix`, with CPU, CUDA, and WebGPU implementations ([#32687](microsoft/onnxruntime#32687)). - Extended `GatherBlockQuantized` to FP8/FP4 data, full-axis blocks, and scale broadcasting, and improved CPU UINT8 dequantization. See the out-of-range-index behavior change above ([#32480](microsoft/onnxruntime#32480), [#32621](microsoft/onnxruntime#32621)). ### Plugin Execution Provider Infrastructure - Added support for statically linked plugin EPs, with a static WebGPU plugin build configuration ([#32395](microsoft/onnxruntime#32395)). - Preserved caller streams in single-tensor plugin data transfers and handled null plugin allocator returns ([#32666](microsoft/onnxruntime#32666), [#32683](microsoft/onnxruntime#32683)). - Deduplicated AutoEP custom-op domains and corrected EP assignment reporting after NHWC layout transformation ([#32711](microsoft/onnxruntime#32711), [#33184](microsoft/onnxruntime#33184)). - Added reference guidance for compiled-model compatibility validation, including multi-device scenarios ([#29168](microsoft/onnxruntime#29168)). ## Execution Provider Updates ### CUDA EP CUDA updates include sparse attention and packed continuous batching, expanded paged speculative decoding, INT2 and mixed-width QMoE execution, opt-in MatMul auto-tuning, improved MXFP4 support on Blackwell consumer GPUs, and reliability fixes. ... (truncated) Commits viewable in [compare view](microsoft/onnxruntime@v1.30.0...v1.31.0). </details> Updated [Swashbuckle.AspNetCore](https://github.com/domaindrivendev/Swashbuckle.AspNetCore) from 10.2.3 to 10.3.0. <details> <summary>Release notes</summary> _Sourced from [Swashbuckle.AspNetCore's releases](https://github.com/domaindrivendev/Swashbuckle.AspNetCore/releases)._ ## 10.3.0 ## What's Changed * Change nullability handling for schemas with `anyOf`, `allOf`, `oneOf` or `type = null` by @ldeluigi in domaindrivendev/Swashbuckle.AspNetCore#3976 * Handle leading slash on route prefix during index.html GET by @superstator in domaindrivendev/Swashbuckle.AspNetCore#4084 * Move NRT handling to DataContractResolver by @desjoerd in domaindrivendev/Swashbuckle.AspNetCore#3998 * Clarify nullable behaviour by @martincostello in domaindrivendev/Swashbuckle.AspNetCore#4094 * Fix review issues for caching and encoding by @martincostello in domaindrivendev/Swashbuckle.AspNetCore#4093 * Support multiple response types per status code by @DeagleGross in domaindrivendev/Swashbuckle.AspNetCore#4100 * Add coverage for `AddServer` across multiple documents by @RaphaelFakhri in domaindrivendev/Swashbuckle.AspNetCore#4115 * Fix Swagger sort key fallback for empty endpoint tags metadata by @AleksaMCode in domaindrivendev/Swashbuckle.AspNetCore#4125 * Fix parameter matching to consider location when names collide by @nonsoikem in domaindrivendev/Swashbuckle.AspNetCore#4126 * Emit form encoding for properties of a complex form parameter by @RaphaelFakhri in domaindrivendev/Swashbuckle.AspNetCore#4128 * Keep a property's own summary when comments come from several XML files by @RaphaelFakhri in domaindrivendev/Swashbuckle.AspNetCore#4129 * Strip query string from relative document URLs in Swagger UI by @GabeSilvaDev in domaindrivendev/Swashbuckle.AspNetCore#4152 * Document that System.Text.Json ignores `[EnumMember]` by @RaphaelFakhri in domaindrivendev/Swashbuckle.AspNetCore#4127 * Drop the regex route constraint from the default Swagger route by @RaphaelFakhri in domaindrivendev/Swashbuckle.AspNetCore#4130 * Serialize custom values in the UI `ConfigObject.AdditionalItems` by @RaphaelFakhri in domaindrivendev/Swashbuckle.AspNetCore#4132 * Use Minimal APIs JSON options when MVC is not registered by @RaphaelFakhri in domaindrivendev/Swashbuckle.AspNetCore#4116 * Resolve enum contracts against the underlying type under source generation by @RaphaelFakhri in domaindrivendev/Swashbuckle.AspNetCore#4131 ## New Contributors * @ldeluigi made their first contribution in domaindrivendev/Swashbuckle.AspNetCore#3976 * @superstator made their first contribution in domaindrivendev/Swashbuckle.AspNetCore#4084 * @desjoerd made their first contribution in domaindrivendev/Swashbuckle.AspNetCore#3998 * @RaphaelFakhri made their first contribution in domaindrivendev/Swashbuckle.AspNetCore#4115 * @AleksaMCode made their first contribution in domaindrivendev/Swashbuckle.AspNetCore#4125 * @nonsoikem made their first contribution in domaindrivendev/Swashbuckle.AspNetCore#4126 * @GabeSilvaDev made their first contribution in domaindrivendev/Swashbuckle.AspNetCore#4152 * @JulianThoms made their first contribution in domaindrivendev/Swashbuckle.AspNetCore#4182 **Full Changelog**: domaindrivendev/Swashbuckle.AspNetCore@v10.2.3...v10.3.0 Commits viewable in [compare view](domaindrivendev/Swashbuckle.AspNetCore@v10.2.3...v10.3.0). </details> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This was referenced Oct 13, 2026
Open
mdariynislam
approved these changes
Oct 18, 2026
This was referenced Oct 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Promote the model package surface from the experimental name-based lookup to the stable
OrtModelPackageApitable.OrtApi::GetModelPackageApiwith stable C and C++ APIs.lintrunnerpassed. Build and test execution were not run locally at the author's request.Motivation and Context
The model package API has completed its experimental phase and is ready to become a stable, versioned ONNX Runtime API surface.