Skip to content

gguf-py : add numpy Q1_0 and Q2_0 de/quantization support - #30630

Merged
CISC merged 2 commits into
ggml-org:masterfrom
SomeNobody21112:gguf-py-q1_0-q2_0
Oct 16, 2026
Merged

CISC merged 2 commits into
ggml-org:masterfrom
SomeNobody21112:gguf-py-q1_0-q2_0

Conversation

@SomeNobody21112

@SomeNobody21112 SomeNobody21112 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Overview

This adds two classes Q1_0 + Q2_0 to gguf-py/gguf/quants.py, so now numpy Q1_0 / Q2_0 support in quants.py, so gguf.dequantize / quantize work for them. Closes #30626.

Additional information

  • Tested the Python code against the C code using the inbuilt script (test_quants.py) and got an exact match 66/66 across 27 types. So the new ones are right and nothing broke.
  • pytest seven passed
  • Q1_0's scale is computed with np.cumsum not np.sum, since C adds the 128 values one by one and np.sum adds them pairwise and float rounding then differs. np.sum gave a different scale than C in 18 of 65,536 blocks and np.cumsum gave zero.
  • setup: Windows 11, gcc 16, Python 3.11, numpy 2.4

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES, AI was used to write the code and test runs but I manually reviewed the codes and verified them.

@SomeNobody21112
SomeNobody21112 requested a review from CISC as a code owner October 15, 2026 21:44
Comment thread gguf-py/tests/test_quants.py Outdated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CISC
CISC merged commit 495c580 into ggml-org:master Oct 16, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: gguf-py: add numpy quantisation for Q1_0 and Q2_0

2 participants