Repository navigation
Add Clef-Flash joint schema decision model support - #760
Draft
gramalingam wants to merge 3 commits into
Draft
gramalingam wants to merge 3 commits into
gramalingam wants to merge 3 commits into
Conversation
Export the pinned Cloudflare checkpoint as vision, embedding, full-record backbone, and joint decision-head components. Preserve publisher schema spans and option probabilities, stream backbone and fused-QKV head weights, and reject autoregressive runtime configurations. Match nested Qwen3.5 rotary settings and DeltaNet precision, pool long schema spans in float32, and cast processor pixels to the vision weight dtype. Add publisher parity, streamed multimodal pipeline, revision, loader, CLI, and metadata coverage plus documentation and a text inference example. Full-size real-checkpoint and accelerator inference remain explicitly unverified. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>
|
|
Performance Comparison
|
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
🏗️ Architecture Diff
qwen3_5_vl (hybrid-qwen-vl) / vision_encoder — 35 change(s)Op summary: 251 → 252 nodes --- base
+++ head
@@ -1,3 +1,4 @@
+Cast
Reshape
Conv
ReshapeAdded nodes:
Modified attributes:
Connectivity changes:
Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
Record the dedicated four-component Clef L1-L3 tests in the coverage audit exemption list. The generic causal-LM/config harness cannot represent schema span inputs or load the separate head sidecar; keep the real-checkpoint golden limitation explicit instead of claiming generic coverage. Sort Qwen vision imports using the CI-pinned Ruff 0.16.2 rules. Validate the coverage audit and complete Clef suite, plus lint and formatting across all PR Python changes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Cross-dtype fused-QKV serialization can fail, and registry coverage metadata is incomplete.
3 open findings
What changed in this PR
Adds Clef-Flash as a four-component multimodal structured-decision pipeline.
Changes:
- Adds backbone, vision, embedding, decision-head export and weight streaming.
- Adds host-side schema encoding and runtime contracts.
- Adds tests, documentation, and an ONNX Runtime example.
| File | Description |
|---|---|
tests/build_graph/_support.py |
Exempts specialized Clef I/O. |
src/mobius/tasks/_vision_language_3model.py |
Casts processor pixels to model dtype. |
src/mobius/tasks/_clef.py |
Builds the four Clef components. |
src/mobius/tasks/__init__.py |
Registers the Clef task. |
src/mobius/models/qwen35.py |
Broadens embedding model typing. |
src/mobius/models/clef.py |
Implements Clef configuration, head, and model. |
src/mobius/models/clef_test.py |
Adds graph, parity, loader, and runtime tests. |
src/mobius/models/__init__.py |
Exports ClefFlashModel. |
src/mobius/integrations/transformers/_clef.py |
Detects and streams Clef checkpoints. |
src/mobius/integrations/transformers/_builder.py |
Dispatches Clef before Qwen3.5. |
src/mobius/integrations/ort_genai/auto_export.py |
Rejects unsupported ORT GenAI export. |
src/mobius/integrations/onnx_genai/auto_export.py |
Emits advisory component metadata. |
src/mobius/integrations/clef.py |
Encodes schemas and restores labels. |
src/mobius/_registry.py |
Registers the Clef architecture. |
src/mobius/__main__.py |
Routes Clef runtime metadata correctly. |
README.md |
Lists structured-decision support. |
examples/clef_flash.py |
Demonstrates text-only inference. |
docs/model-catalog.md |
Adds Clef to the catalog. |
docs/index.md |
Includes the Clef guide. |
docs/clef-flash.md |
Documents export, inference, and limitations. |
🧠 Review effort: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Cast each fused attention QKV split to the target initializer dtype before the lazy streaming loader materializes it. Cover all nine float32/float16/bfloat16 source-target combinations for weights and biases through save/reload. Run multimodal pipeline parity on reloaded exported packages rather than original lazy bindings, and consistently import the public build function in Clef tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add support for
Cloudflare/clef-flashas a full-record structured decision model rather than an autoregressive language model.fde727a287004204b7518dcc983fe64379776712and forward revisions across assets.Validation
RUF105, so equivalent rules were run without that selector.Limitations and waivers
Keeping this PR in draft pending full-size checkpoint and accelerator validation.