Support pruned expert variants (GSQ-RCO-Coder: 256 of 512 experts)

[pjgmobile]

Support pruned expert variants (GSQ-RCO-Coder: 256 of 512 experts)

Motivation

ISTA-DASLab recently published Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF — the same
architecture with half the routed experts pruned away per layer (512 → 256,
qwen4exp.expert_count = 256, still 10 active). The engine currently specialises to the
canonical geometry in a number of places, so a pruned model fails at a different spot for
each one: the pack tool divides by a hardcoded 512, the pack loader validates against the
compile-time default, the runtime graph is default-constructed, the prefill kernels and
buffers assume 512, and serve mode additionally demands a canonical profile-filled cache.

This PR threads the model's own declared expert geometry through those paths so pruned
variants boot and run, while canonical behaviour stays byte-identical.

Changes

• tools/iq_pack.py — N_EXPERT = 512 was hardcoded; the expert count is now derived
  from the router tensor's shape (ffn_gate_inp.weight, dim 1) with a uniformity check
  across layers. Blob sizes, offsets and the native_experts.txt header all follow it.
• src/kernels/cpu/expert_layout.cpp — expert_layout_load() trusted the caller's
  geometry (a compile-time default) for the contiguity stride. It now parses
  n_expert from the pack header (v3 headers already record it) and uses that for both
  the stride and the stored layout, so ArenaExpertSource's geometry check passes.
• include/strata/artifact/gguf_reader.hpp — Qwen4ExpGuard validated every GGUF
  against canonical constants. experts/experts_used now default to 0 = presence-only;
  block count, hidden size and head counts are still hard-validated.
• src/program/generate.cpp — the runtime ModelGeometry was default-constructed;
  it now reads qwen4exp.expert_count / expert_used_count from the --native shard
  (K too). The MTP drafter deliberately keeps the canonical geometry via a
  static const ModelGeometry — the draft layer is the original 512-expert head, and it
  sizes its own arena from it (speculation works unchanged; acceptance 74-85% measured).
  The serve-mode gate also no longer demands the profile-filled graph-hit path: an
  on-demand (--expert-cache auto) cache is a valid serve configuration.
• src/core/layer.cpp — the native fused router stays canonical-only; non-canonical
  geometry falls back to the generic router_top10 instead of erroring.
• src/prefill/* — the NE = 512 constant is replaced by the geometry's expert
  count in buffer sizing, host loops and residency indexing; route_kernel is
  templatized on registers-per-lane (512 → 16, 256 → 8, anything else takes the generic
  router_top10); route() gained an n_expert parameter; moe_set_bytes() likewise.

Design notes

• The model file is the authority on its own MoE shape; the compile-time defaults remain
  as fallbacks and for the canonical fast paths.
• The draft/target split is intentional: pruning changes the target's routed experts but
  the shipped MTP draft is still the original head, so it must keep 512x10 sizing.
• Profiles: the shipped data/expert-profile.bin is 48x512. For a pruned model we
  remapped the ranked (layer, expert) pairs onto the survivor indices using the release's
  published tensor-allocation/*.rco-allocation.txt (Section 2 records each survivor's
  original index) — 8000 → 5492 pairs survived remapping, trimmed to fit VRAM. A
  make_profile.py run against pruned-model routing would do this properly; happy to
  follow up with that as a companion tool.

Testing (Ryzen 5 5600X, RTX 3090 24 GB, CUDA 13.1)

• Pack built from the IQ1_M pruned release (shard 2 is byte-identical to the canonical
  n-gram table; symlinked, no re-download).
• Pruned lane boots: 11 pool workers, 262144 ctx, KV int8, remapped profile
  (4500 slots), MTP draft over the canonical head.
• ~3.5-hour real agentic coding session through llama-swap (three.js scene build):
  peak 192k tokens, one omp compaction, zero engine errors, decode 27-42 tok/s
  (post-0.1.14 AVX2 i-quant kernels: 38.7 cold / 42.6 warm on fixed prompts vs 23.1/24.6
  before this upgrade), draft acceptance 65-85%.
• Canonical lanes unaffected: Q2_0 daily driver answers at its usual speed (53.6 tok/s
  spot check), IQ2_XS and vision configs untouched.

Disclosure

This PR was authored by GLM-5.3-Flash (ZCode agent); Pete (pjgmobile) tested it on real
workloads overnight and submitted it.

[Niko1221]

Thank you @pjgmobile - this is exactly what the Coder needs, and it reads cleanly: the model file as the authority on its expert count, the canonical fast paths untouched, and the MTP draft deliberately kept on the 512-expert geometry (the right call - the shipped draft layer is the original head).

I've merged it into the branch for the next release and built on it:

• Expert profile: data/expert-profile-coder.bin ships with it - the full 48 x 256 ranking: the shipped profile mapped through section 2 of the release's rco-allocation.txt (5,492 of the top 8,000 survive, as you found), then every other kept expert. With 2,537 slots on a 12 GB card, 72% of the expert reads of a 4K code prompt hit the GPU (a wrong index mapping would give about the slot share, ~21%), so the mapping holds. tools/make_profile.py also takes --n-expert now.
• Setup: START-HERE.bat --setup --family coder. Its shard 2 and vision encoder are byte-identical to the original's, so with the original installed only the 29.6 GB shard 1 is downloaded, and 32 GB of RAM is enough to install it.
• Measured here (RTX 5070 12 GB, Ryzen 5 7600, 64 GB, the settings setup writes), 1K / 4K / 32K / 64K / 128K / 262K: prompts 599 / 1,152 / 1,298 / 1,350 / 1,266 / 1,034 tok/s, output 53 / 51 / 53 / 51 / 44 / 43 tok/s, answers coherent at every length. Its experts are stored as IQ2_S-IQ4_XS / IQ4_NL / Q2_0 ("IQ1_M" is just the release name), so the MMQ prompt path and the AVX-512/AVX2 row kernels all apply - and with half the experts the whole 23 GB arena pins and a 12 GB card holds twice the share of them.
• Images work with the original's encoder (it read a Monitor screenshot's numbers correctly). The experimental speed projection loads and runs on it too.
• The original models are unchanged by the PR (Q2_0 and IQ3_S at 4K and 32K within run-to-run noise); it also builds on Linux.

It goes out with the next release, and the README gets the Coder's row - thanks again for the overnight testing.

[Niko1221]

Released in v0.1.16: START-HERE.bat --setup --family coder installs the Coder (with the original installed, only its 29.6 GB shard 1 is downloaded), and the README has its row. Thanks again!