You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 99d3155
Browse filesBrowse the repository at this point in the historyBrowse files
## Implementation
- Adds architecture-discriminated `VibeVoiceForASRStreamingTraining`
support for both official Microsoft streaming-ASR checkpoints, without
changing the existing VibeVoice TTS route:
-
`microsoft/VibeVoice-ASR-Streaming-1.5B@4262d23d8a539a6530cf64fbd0b1751ef9a30853`
-
`microsoft/VibeVoice-ASR-Streaming-7B@60d858b518b4e19d404af3737f848fc185b30177`
- Exports the executable ASR pipeline as three truthful stages:
`audio_encoder` (acoustic and semantic causal tokenizers plus
connectors), `embedding` (speech embedding replacement), and `decoder`
(Qwen2 with explicit KV caches).
- Models both causal convolution cache sets and final-chunk stride
padding as ONNX inputs/outputs. Streaming control tokens
(`<speech_start>`, `<speech_end>`, forced `<|text_chunk_end|>`), chunk
loop/sampling, hotword prompt assembly, and speaker JSON parsing remain
host-owned.
- Declares the decoder attention mask arbitrary through the generic
`mobius.attention.requires_arbitrary_mask` contract, preventing
incorrect GQA fusion for left-padded ASR batches.
- Routes all 901 executable checkpoint tensors exactly once for each
variant and explicitly excludes the 276 `acoustic_tokenizer.decoder.*`
training-VAE tensors that are not on the ASR inference path. The 1.5B
tied embedding/head checkpoint preserves its explicit `lm_head.weight`.
- Emits graph-derived advisory metadata and processor assets, but marks
ORT GenAI runtime orchestration unsupported rather than publishing a
nonfunctional runnable claim.
## Provenance and runtime boundary
- Executable reference:
`microsoft/VibeVoice@505653d3873b065a488aea551c6ee3dc51d3062f` with
`transformers==4.51.3`; ASR-critical blobs were checked unchanged
through upstream `1541f590c7099820f10ea012f48d2399282df69f`.
- The dedicated source-parity job force-installs the exact Transformers
4.51 ABI (`huggingface-hub==0.31.4`, `tokenizers==0.21.4`) after the
public Transformers 5 environment.
- No local real-weight downloads were attempted, so this PR makes no
L4/L5 real-transcription claim. `onnxruntime-ep-mlx 0.29.5` / ORT 1.29
cannot execute the complete VibeVoice decoder due to `CumSum cannot
infer output shapes`; this blocks local MLX real-runtime evidence.
## Validation
- L1 staged graph/cache/role and arbitrary-mask-fusion-exclusion tests.
- Full pinned L2 builds and complete 1,177-tensor safetensors-index
audits for both variants.
- Exact-source synthetic two-chunk prefill/cached-decode parity plus
both pinned processor contracts: 24 kHz framing, left-padding,
English/Chinese hotwords, and speaker JSON normalization.
- Full non-integration suite: `4730 passed, 56 skipped, 1 subtests
passed`.
- `lintrunner f --output oneline --all-files` and diff checks.
- Fresh independent exact-head GPT-5.6 Terra review: no high-confidence
findings.
---------
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4ef52c4-0459-4efa-bc06-d0ebca2275f2
0 commit comments