[bong-water-water-bong]

W4A4 prefill +6-10% for free: llama-bench pp2048 2,001 -> 2,158, pp16384 1,227 -> 1,283 (ub 2048 is no better, decode unchanged). Through 1bit serve --long-model, the 16K first token comes at 12.96 s instead of 13.89 s. tests/hadamard_route.sh checks the flag. Also measured: flash attention off drops pp16384 to 337, so llama-server's default -fa auto (on) must stay.

🤖 Generated with Claude Code

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit 98263cf

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Incorrect micro-batch size for Hadamard-rotated models The PR introduces a hardcoded -ub 1024 for Hadamard-rotated Q4_0 models, but this value should be configurable or validated against the actual hardware capabilities. The performance gains claimed (6-10% for prefill) are specific to certain models and conditions, and the micro-batch size may not be optimal for all hardware configurations or use cases. // 1024-token micro-batches: prompt processing +3-8% over the default 512 (Qwen3-Coder-30B-A3B, // pp2048 2,001 -> 2,158, pp16384 1,227 -> 1,283; 2048 is no better), decode unchanged argv.insert(argv.end(), {"-ub", "1024"}); Unverified performance claims The documentation states performance improvements of 6-10% for prefill with -ub 1024, but these numbers are not verified for all hardware configurations or model types. The claim should be supported by measurements across different setups to ensure correctness. side 2-5% against the W4A4 row. A rotated file now runs 1024-token micro-batches (`-ub 1024`), which raises the W4A4 prefill 6-10% (server-reported 2,217 / 1,742 / 1,295 tok/s at 2K / 8K / 16K, from 2,000 / 1,607 / 1,221): a 16K-token prompt's first token at 12.96 s instead of 13.89. Both files stay loaded (here 17 + 16 GiB). It needs the Vulkan