Skip to content

pr-set: pin CUDA IQ1_M MMQ (#304) - #305

Open
danielhanchen wants to merge 1 commit into
masterfrom
pr-set-iq1m-mmq
Open

danielhanchen wants to merge 1 commit into
masterfrom
pr-set-iq1m-mmq

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

Pins #304 (CUDA: add IQ1_M to MMQ) at 64b2a7e. Nothing upstream broke; this adds a feature. IQ1_M had no MMQ kernel, so batched IQ1_M matmuls (prompt processing, and MoE MUL_MAT_ID in particular) fell back to dequantize + cuBLAS. No equivalent PR is open on ggml-org/llama.cpp.

Evidence

llama_pinset.py sim with this pin set:

tag merge pin_contract merge_checks
b11756 (aged base now) clean, 21 pins all intact clean
b11757 clean all intact clean
b11759 clean all intact clean
b11760, b11761 ggml-org#25731 REFUSED 1 pin not intact clean

The b11760+ refusal is ggml-org#25731 (Inkling) and is the same on the current master pin set without this change; #304 merges cleanly there too.

llama_pinset.py verify --cuda on b11756 and b11759 (B200):

check b11756 b11759
CPU gate build + feature_matrix CPU ok, 13/13 ok, 13/13
full CUDA build ok ok
test-chat pass pass
test-backend-ops MUL_MAT / MUL_MAT_ID 2582/2582, 1027/1027 2582/2582, 1027/1027
test-backend-ops PAD / GET_ROWS 37/37, 349/349 37/37, 349/349
feature_matrix --gpu 13/13 13/13
new iq1m-mmq check, --gpu 12/12 + 16/16 matched CPU 12/12 + 16/16 matched CPU

audit --pr-set: #304 ok (base base/upstream-d80619397, GitHub diff == the pin's own change, +256/-2 in 16 files); feature-checks coverage ok. The FIX rows for #264, #266 and #298 are the same on the current master pin set.

Kernel correctness, KLD vs the CPU backend and the llama-bench numbers (about 3x prompt processing on an IQ1_M MoE, decode unchanged) are in #304.

Not covered

  • ROCm, Vulkan, macOS, Windows and ARM legs only run in the nightly.
  • Tags newer than b11761 were not simmed.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T00:21:30.308956Z eb114cf PR opened
🔒 Security Review ✅ Completed 2026-10-21T00:24:32.906798Z eb114cf PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant