feat(gguf): serve Qwen3.5 dense and MoE GGUF checkpoints

[jason-fxz]

Summary

Serves llama.cpp qwen35 (dense) and qwen35moe (MoE) GGUF files, which cover Qwen3.5, Qwen3.6 and Qwen3.8 (27B and 2.4T-A95B), on the existing qwen3_5_moe model classes. Text only; MTP blocks in the file are skipped.

Changes

• models/qwen3_5_moe/gguf.py (new):
  • Builds the config from the GGUF metadata.
  • Maps checkpoint names to GGUF tensors, including the fused ffn_gate_up_exps.
  • Undoes the converter's transforms: GDN value heads go from tiled back to grouped order by reordering whole packed rows; A_log is restored from -exp(A_log).
  • Refuses prism (Hadamard-rotated) files and NVFP4 GGUF files with per-tensor .scale tensors.
• gdn.py, model.py:
  • ssm_out keeps its tiled input columns, because a K-quant superblock spans two heads. Under GGUF the GDN layer permutes its input to that order instead.
  • embed_tokens takes the quant config, so a quantized token_embd is served packed.
• gguf/config.py, register.py: register the two archs.

Tested models

RTX PRO 6000 Blackwell (sm_120), ft serve --model-path <file>, AIME25 #1 run greedy to EOS, plus 48 staggered requests at concurrency 16.

Model	Quant	bs=1 tok/s	AIME #1
unsloth/Qwen3.6-35B-A3B-GGUF	UD-Q4_K_M, fused / offload	260 / 160	70 / 70
bartowski/Qwen_Qwen3.6-35B-A3B-GGUF	IQ3_XXS	264	70
unsloth/Qwen3.8-27B-GGUF	UD-Q4_K_M	79	70
unsloth/Qwen3.5-9B-GGUF	Q4_K_M	216	70
DavidAU/Qwen3.5-9B-…-NEO-IMATRIX-MAX-MTP-GGUF	NEO-MAX-MTP-Q4_K_M	185	70
unsloth/Qwen3.5-4B-GGUF	UD-Q4_K_XL	317	70
ggml-org/Qwen3.5-0.8B-GGUF	BF16	550	wrong (0.8B)

Staggered requests: no corrupted output on any file.

Results

• Weights: after ggml.dequantize, every float tensor is bf16-exact against the HF checkpoint, and Q8_0 tensors reach cos 0.99998. Dry load against the meta-built model matches 493/493 params (35B-A3B) and 738/738 (27B).
• vs llama.cpp f2cc728: same file, greedy, 6 prompts × 128 tokens. Identical outputs: 0.8B 4/6, 9B 5/6, 35B 3/6, 27B 1/6. The divergences checked are near-ties in llama.cpp's own probabilities (e.g. 0.509 vs 0.488).
• Coverage: header check (dry load, kernel coverage, tokenizer, chat template) of 1204 files from 73 HF repos: 1100 pass. Every dense size passes, and so does every Q2_K–Q8_0, IQ2–IQ4, BF16 and MXFP4_MOE quant.
• Unit tests: tests/moe, tests/kernels/test_ggml.py, and the tests/models tests for qwen3_5_moe, the registry, the loader and gemma4 GGUF pass.

Not covered

• IQ1_M weights (no MMQ kernel).
• Expert layers that mix float and quantized types (follow-up PR).
• Experts whose gate and up types differ.
• ik_llama.cpp types, NVFP4 GGUF, prism.