metal: TQ1_0 kernel performance (dense decode +39-55%, prefill +78-90%)

[bri-prism]

Two performance changes to the Metal TQ1_0 kernels. Stacked on feat/tq1_0-cuda-v7, which carries the kernels themselves and has no PR of its own yet, so the base here is that branch rather than prism-v7. Relates to #141.

Measured on an M5 Pro with llama-bench -p 512 -n 128 -r 3, across four ternary models: three dense, spanning roughly a five times parameter range, plus one mixture-of-experts. Three build arms were used, each in its own worktree, with the embedded shader confirmed distinct before every run.

model	decode	prefill
dense, smallest	+45.7%	+82.6%
dense, middle	+39.1%	+78.4%
dense, largest	+55.2%	+89.7%
mixture-of-experts	+21.8%	+112%

The decode gain is consistently larger on the dense models, 39 to 55 percent, than on the MoE at 21.8 percent. Dense decode is dominated by the mat-vec kernel this change touches, whereas the MoE spends a meaningful share of its decode in routing and gather work that dilutes the improvement. Prefill improves substantially everywhere.

For scale on the smallest dense model, where a same-binary TQ2_0 comparison was also taken: TQ1_0 moves from 53 percent of TQ2_0 prefill and 61 percent of its decode, to 96.5 percent and 88.8 percent respectively, at 82 percent of the file size. TQ2_0 still wins both axes, so this is now a size against speed trade rather than TQ1_0 being uncompetitive.

The two changes are independent and touch different paths, so they are separate commits.

1. Flatten the mat-vec into two uniform passes (decode)

The mat-vec walked a block's payload as three regions of 32, 16 and 4 bytes, each behind its own lane predicate. The second and third passes leave most of the simdgroup idle while the group still waits for them, so only 52 of 96 lane-slots do useful work.

qs[48] and qh[4] are adjacent in block_tq1_0, so the payload is really a flat 52 bytes. Lane l now takes byte l and byte 32 + l, covering all 52 in two passes at 52 of 64 slots.

The obstacle is that the tails carry different digit counts, five per byte for qs and four for qh, so running them together would reintroduce the divergence being removed. Padding the qh tail's fifth activation with zero makes the five-digit collapse reproduce the four-digit one exactly: the g_5 coefficient becomes zero and g_4 becomes y_3, which is the four-digit form. Both tails then share one code path. Lanes past the payload get zero coefficients and a clamped index, so they read in bounds and add nothing.

2. Remove the per-element integer work from the dequantize (prefill)

Prefill reaches TQ1_0 through the element-indexed dequantize template, which recomputed a modulo, a divide and an integer multiply-shift for every one of the sixteen elements it produced. On an ISA that cannot co-issue integer and floating-point work, that dominated.

Those sixteen elements share a digit index. For il < 10 they are sixteen consecutive bytes of the 32-byte region at digit il/2; for 10 <= il < 15, sixteen consecutive bytes of the 16-byte region at digit il-10; only il == 15 walks the qh tail. Hoisting that leaves two loop-invariant constants, and the digit comes from the same exact-float identity the mat-vec already uses, so the inner loop is a floor pair with no integer arithmetic.

Note this does not fix the underlying five times byte re-read, which is inherent to the sixteen-contiguous-element interface. A block-at-a-time unpack into the shared-memory tile is still available on top of this.

Correctness

Both digit identities were checked against the integer reference before the kernels were touched: the zero-padded collapse over all 256 byte values, and the rewritten dequantize over 77824 elements covering every il including extreme bytes. test-backend-ops on Metal passes 17 MUL_MAT, 7 MUL_MAT_ID and 4 GET_ROWS tq1_0 cases with zero failures after each change.

The dequantize change is bit-identical: per-element values are unchanged and nothing is repartitioned, and greedy generation matches the previous commit exactly apart from the build stamp.

The flatten change is not bit-identical, and should not be expected to be. It moves the qh tail off lanes 0-3 and onto lanes 16-19, so the simdgroup reduction sums a different partition of the same terms and fp32 reassociation shifts the last bits, by about the magnitude the existing five-digit path already carries. The transform is algebraically exact; only the reduction order moves. On the degenerate bench model used here, whose top-two logits are nearly tied, greedy decoding follows the same tokens for a short prefix and then diverges.

An earlier revision of this description claimed byte-identical greedy output for the flatten change. That was measured against a baseline build directory whose Metal dylib had since been rebuilt from modified sources, so it compared the change against itself. The claim is withdrawn.

Measuring this yourself

Two traps, both of which produced wrong answers here before being caught.

GGML_METAL_PATH_RESOURCES is only consulted when no precompiled metallib exists. This build logs using embedded metal library, meaning the shader is baked into libggml-metal.dylib, so pointing that variable at two source trees changes nothing. Appending #error to one tree and still getting a clean run is what exposed it.

Separate build directories in one source tree are not sufficient either. Building any target in a directory re-derives its dylib from whatever the sources currently say, so a baseline verified at setup can be silently overwritten later. Use separate worktrees, and re-confirm with strings <dylib> | grep -c <marker> immediately before each measurement rather than once. The build banner is not provenance: it records the commit at configure time and will report a clean SHA for a binary containing uncommitted changes.

Not applicable

fp16 for the digit extraction does not work, recorded so nobody retries it. The method needs floor(3^k * b / 256) exact; at 3^5 * 255 = 61965 the fp16 spacing is 32, so consecutive integers are not representable. Measured over all bytes it gets 17 of 1280 digits wrong, for example byte 79 digit 3 yields 1 where the reference gives 0, a wrong ternary level rather than a rounding difference. Resolving the 1/256 fractional steps needs roughly 16 mantissa bits and fp16 has 11, so even the first digit fails.

[Copilot]

🔵 Needs a closer look

Low-level kernel changes warrant final human review; remaining findings are non-blocking style nits.

Pull request overview

Optimizes Metal TQ1_0 decode and prefill kernel performance.

Changes:

• Flattens mat-vector payload processing into two uniform passes.
• Hoists dequantization indexing and removes per-element integer work.

File summaries

File	Description
ggml/src/ggml-metal/kernels/mul_mv.metal	Improves SIMD utilization; comment formatting nits remain.
ggml/src/ggml-metal/kernels/dequantize.h	Optimizes digit extraction; a comment concision nit remains.

Review details

Suppressed comments (1)

ggml/src/ggml-metal/kernels/mul_mv.metal:3175

• This explanation is hard-wrapped across five lines despite the repository requirement for concise, non-wrapped comments. The algebraic invariant can be preserved in two sentences.

            // The qh tail carries 4 digits per byte rather than 5. Padding its fifth
            // activation with zero makes the five-digit collapse reproduce the
            // four-digit one exactly (the g_5 coefficient becomes 0 and the g_4 one
            // becomes y_3), so both tails run the same code and the pass does not
            // diverge. Lanes past the payload keep all-zero activations and add zero.

• Files reviewed: 2/2 changed files
• Comments generated: 2
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.