Skip to content

pr-set: pin #303 (CUDA FA MMA in-kernel q4_0/q8_0 K/V dequant for 3-16 tokens) #306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
danielhanchen wants to merge 3 commits into
base: master
Choose a base branch
from
Open

pr-set: pin #303 (CUDA FA MMA in-kernel q4_0/q8_0 K/V dequant for 3-16 tokens) #306

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7dd9b58
pr-set: pin #303 (CUDA FA MMA in-kernel q4_0/q8_0 K/V dequant for 3-1…
danielhanchen Oct 21, 2026
0f4a294
pr-set: pin #303 at 34a66f42b (composes after ggml-org#25731), repin …
danielhanchen Oct 21, 2026
34acf7b
Merge branch 'master' of https://github.com/unslothai/llama.cpp into …
danielhanchen Oct 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions scripts/unsloth/feature-checks.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,12 @@
{ "kind": "backend-op", "op": "GET_ROWS", "params": "type=mxfp8" },
{ "kind": "mtmd", "projector": "deepseek41v" }
]
},
"cuda-fa-quant-small-batch": {
"owner": "unslothai#303",
"checks": [
{ "kind": "backend-op", "op": "FLASH_ATTN_EXT", "params": "n_seq=4" }

@chatgpt-codex-connector chatgpt-codex-connector Bot Oct 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Assert the optimized CUDA path in the feature check

In the GPU feature-matrix run, this probe only checks that FLASH_ATTN_EXT produces correct results for some n_seq=4 cases. If dispatch regresses to the previous whole-cache f16 conversion path, the outputs remain correct and all matched cases still pass, even though the performance feature carried by this pin is no longer active. Add evidence that the q4_0/q8_0 MMA kernel was selected and that conversion kernels were absent, rather than treating numerical equivalence as proof of this optimization.

Useful? React with 👍 / 👎.

]
}
},
"unchecked": {
Expand Down
3 changes: 2 additions & 1 deletion scripts/unsloth/pr-set.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
"https://github.com/unslothai/llama.cpp/pull/298/commits/d1781c3ea6ab7c75c635a8fea60d2b3f4937c645",
"https://github.com/unslothai/llama.cpp/pull/304/commits/64b2a7e7bdddc0c00de67418b4b056cad60666d6",
"https://github.com/unslothai/llama.cpp/pull/311/commits/819e95b316743cd2ed7bb54b68f52f5ca1ea820a",
"https://github.com/unslothai/llama.cpp/pull/314/commits/08f367a2a56a7b4405b106eb23a227a910571786"
"https://github.com/unslothai/llama.cpp/pull/314/commits/08f367a2a56a7b4405b106eb23a227a910571786",
"https://github.com/unslothai/llama.cpp/pull/303/commits/34a66f42be7b0624b00e039725ea879fd772b14f"
]
}