Repository navigation
Conversation
|
Tommaso Adani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Performance Comparison
|
🏗️ Architecture Diff
No architecture changes detected. ✅ Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Standardize symbolic dimension names used by Mobius's common ORT GenAI decoder
exports.
The updated decoder paths now consistently use:
batch_sizesequence_lengthpast_sequence_lengthtotal_sequence_lengthThis applies to:
inputs_embedsdecoder builder used by multimodal andspeech-language packages
The tests save each graph as ONNX, load it again, and verify the serialized
input/output dimension names. This ensures the test covers the interface seen
by downstream consumers rather than only Mobius's in-memory graph.
Motivation
Olive's discrepancy check needs to convert symbolic ONNX dimensions into
concrete shapes when creating test inputs.
Mobius's common decoder paths previously emitted several names for the same
concept, including:
batchandbatch_sizesequence_lenandsequence_lengthpast_sequence_lenandpast_sequence_lengthThere was also an internal inconsistency where a cache input declared
past_sequence_len, while the attention-mask expression referred topast_seq_len. These are different symbols from an ONNX consumer'sperspective.
As a result, Olive needed Mobius-specific aliases and fallback values to
resolve the exported shapes. Establishing a consistent vocabulary in Mobius
makes the ONNX interface more predictable and moves responsibility for naming
correctness to the exporter.
Changes
CausalLMTask.HybridCausalLMTask.build_decoder_from_embeds.total_sequence_lengthsymbol instead of arithmeticexpression strings.
Validation
src/mobius/tasks/_task_test.pygit diff --checkpassed.A broader graph-construction run completed with 1,379 passed and 48 skipped.
It also reported 14 existing shape-inference/checker failures in specialized
models under the local environment's older ONNX dependency stack. The focused
tests covering the changed decoder paths passed.
Compatibility
This does not change tensor names, ranks, dtypes, or concrete runtime shapes.
Normal inference callers should therefore be unaffected.
It does change symbolic dimension metadata. A downstream tool that explicitly
depends on the previous strings could be affected. This draft PR is intended
to establish whether these symbolic names should be treated as a documented
ORT GenAI export contract.
Previously exported ONNX artifacts are not modified by this change.
Follow-ups
ORT GenAI owners.
to the same vocabulary.
generation dimensions before saving an ORT GenAI package.
values with explicit discrepancy-check inputs or derived values.
contract is released and Olive's compatibility policy is established.