Skip to content

common : fix -hf tag matching inside longer quant names - #31620

Merged
ggerganov merged 2 commits into
masterfrom
gg/fix-hf-quant-tag-match
Nov 4, 2026
Merged

ggerganov merged 2 commits into
masterfrom
gg/fix-hf-quant-tag-match

Conversation

@ggerganov

Copy link
Copy Markdown
Member

Overview

-hf <repo>:F16 picked the BF16 model when a repo had both, because the tag was matched anywhere in the path: the pattern F16[.-] also matches inside Llama-2-7B-BF16.gguf. The tag now has to start at a token boundary (start of path or after -, ., /), which also stops :Q4_XS from matching IQ4_XS and similar. A tag no file carries now resolves nothing with the usual error listing the available files, instead of silently downloading the wrong dtype.

tests/test-model-resolution.cpp: new F16/BF16 and Q4_XS/IQ4_XS cases, a pinned listing for ggml-org/Llama-2-7B-GGUF which ships both, and 17 pins corrected that had frozen the old behavior. The suite passes on all 172 pinned real listings.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES
    • Harness: dsh:llama.cpp
    • Hardware: Mac Studio, M5 Ultra, 256GB
    • Model: ggml-org/GLM-5.3-Flash-GGUF
    • Stats: ◷ 33.5m · ⚙ 19.3s · ↑ 91k · ↓ 66k · ⇈ 423.6 tok/s · ⇊ 36.1 tok/s · ≡ 112 · ⌨ 12

":F16" picked a BF16 file when the repo had both, since "BF16"
contains "F16". The same applied to ":Q4_XS" vs "IQ4_XS". The tag
now has to start at a token boundary in the path.

Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (Mac Studio, M5 Ultra, 256GB)
@ggerganov
ggerganov marked this pull request as ready for review November 3, 2026 19:42
@ggerganov
ggerganov requested a review from a team as a code owner November 3, 2026 19:43
@github-actions github-actions Bot added the testing Everything test related label Nov 3, 2026

@ServeurpersoCom ServeurpersoCom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, and good to see the real listings harness bite: 17 pins had frozen the F16 -> BF16 bug!

Non-blocking suggestion: the left boundary could also accept _. The harness only pins spec-conformant names so it does not cover this, but some real repos use it as separator, e.g. stepfun-ai/Step-3.5-Flash-GGUF-Q4_K_S ships step3p5_flash_Q4_K_S-00001-of-00012.gguf, where :Q4_K_S would no longer match since an explicit tag has no fallback. (^|[-./_]) would keep it working while still separating F16/BF16 and Q4_XS/IQ4_XS.

Some repos separate the quant with an underscore, e.g.
stepfun-ai/Step-3.5-Flash-GGUF-Q4_K_S ships
step3p5_flash_Q4_K_S-00001-of-00012.gguf, which :Q4_K_S no longer
matched. Add _ to the token boundary and pin the tag and default
lookup cases.
@ggerganov
ggerganov merged commit 4b29d02 into master Nov 4, 2026
15 of 16 checks passed
@ggerganov
ggerganov deleted the gg/fix-hf-quant-tag-match branch November 4, 2026 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants