Skip to content

repack_gguf: refuse the inverted k_b converter instead of rewriting its source - #212

Merged
bong-water-water-bong merged 3 commits into
mainfrom
repack/drop-kb-patch
Sep 29, 2026
Merged

bong-water-water-bong merged 3 commits into
mainfrom
repack/drop-kb-patch

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Follow-up to #209 (review notes on #185/#209). scripts/repack_gguf.py found the deepseek2 converter's old k_b transpose by matching a line of its source text, and rewrote that line in a temp copy. The converter fixed the transpose at the source (5fc52149b), so on a current converter the block never fires, and it would silently misfire if that line ever returned for another reason.

Now: if the converter still contains the inverted transpose, repack stops with ... transposes k_b (16 tiles/head; the forward reads 12). Update the converter to 5fc52149b or later. The Q4_1 config patch next to it is unchanged. 17 insertions, 31 deletions; py_compile passes.

🤖 Generated with Claude Code

…ad of rewriting it

The deepseek2 converter reverted its k_b transpose at the source (5fc52149b).
The engine rewrote the converter's source text to switch the old transpose off,
which silently stops applying if that line changes. Now a converter that still
has it is refused with a message to update it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit 6ccb9a4

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit 6ccb9a4)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

209 - Partially compliant

Compliant requirements:

  • GGUF-on-NPU support for six architectures (qwen3moe, qwen35moe, GLM fixes)
  • Tokenizer and serve fixes
  • NPU serve takes /tmp/1bit-npu-device.lock
  • Repack records GGUF arch for provenance stamp check
  • Accepts Unigram tokenizer.json and strips SentencePiece's dummy-prefix space
  • Registry rows for the six GGUF-on-NPU arches
  • Tests updated with anchor rerun harness

Non-compliant requirements:

  • None

Requires further human verification:

  • None

185 - Partially compliant

Compliant requirements:

  • 1bit serve -m <model.gguf> --device npu repacks GGUF into Q4NX int4 tile format
  • Runs NPU lane end-to-end for six target architectures
  • Answers anchor prompt ("The capital of France is", greedy, max_tokens=1) with " Paris"
  • Performance measured (0.006–0.17 tok/s)
  • Correctness verified across all six arches
  • GLM k_b transpose fix implemented
  • Q4NX directory fast lane stays bit-identical on golden check

Non-compliant requirements:

  • None

Requires further human verification:

  • None
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Incorrect Error Message for k_b Transpose

The error message printed when detecting an inverted k_b transpose in the converter mentions that the forward reads 12 tiles/head, but the actual forward reads 16 tiles/head. This discrepancy could mislead developers trying to debug or fix the issue. The correct value should be 16, matching the number of tiles the converter produces when the transpose is applied.

print(f"repack: {_kb_file} transposes k_b (16 tiles/head; the "
      f"forward reads 12). Update the converter to 5fc52149b or later.")
return 1

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 4bdeb15

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 6ccb9a4

@bong-water-water-bong
bong-water-water-bong merged commit f362a09 into main Sep 29, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the repack/drop-kb-patch branch September 29, 2026 01:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant