Eval bug: aarch64 repack aborts on first token with DeepSeek-V4-Flash (3D MoE expert tensors), workaround --no-repack #26537

Name and Version

version: master 0ab9d6fed (2026-08-02)
also reproduces on 571d0d540 (b10068)
built with GCC 13 for aarch64-linux-gnu, -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release

Operating systems

Linux (Ubuntu 24.04 arm64)

GGML backends

CPU (aarch64, Arm Neoverse-N2, 64 and 96 core machines)

Hardware

Arm Neoverse-N2 server, 64 cores / 251 GB and 96 cores / 672 GB. CPU-only, no GPU.

Models

• ggml-org/DeepSeek-V4-Flash-0731-GGUF (DeepSeek-V4-Flash-0731-MXFP4.gguf, 155 GB)
• unsloth/DeepSeek-V4-Flash-0731-GGUF (UD-Q4_K_XL/*, 155 GB)

Both reproduce.

Problem description & steps to reproduce

On aarch64, llama-server loads all weights successfully and then aborts on the first token with:

ggml/src/ggml-cpu/repack.cpp:4282: GGML_ASSERT(ggml_n_dims(op->src[0]) == 2) failed

Backtrace points through the extra-buffer repack path:

libggml-cpu.so(ggml_cpu_extra_compute_forward+0x60)
libggml-cpu.so(+0x1622c)
libgomp.so.1(GOMP_parallel+0x58)
libggml-base.so(ggml_backend_sched_graph_compute_async+0x474)
libllama.so(llama_context::graph_compute+0x9c)
...
llama_decode

Reproduce:

llama-server -m DeepSeek-V4-Flash-0731-MXFP4.gguf \
  -t 38 -tb 64 -fa 1 --jinja -c 16384 --parallel 1 -ctk f16 -ctv f16 \
  --host 127.0.0.1 --port 8080
# model loads, then aborts as soon as a request is decoded

Workaround: add --no-repack. With it, the same command serves normally and the model performs
well (16.2 t/s decode on 64 cores, 16.6 on 96).

What I think is happening

ggml_n_dims(op->src[0]) == 2 assumes a 2D weight, but DeepSeek-V4's MoE expert tensors are 3D
(experts stacked along a third dimension). The aarch64 repack extra-buffer type appears to accept
the op and then assert inside it, rather than declining to handle a 3D source.

If that reading is right, the fix is for the repack extra-buffer's supports_op (or equivalent) to
return false for sources with ggml_n_dims != 2, so the op falls back to the standard CPU path
instead of aborting. Happy to test a patch on this hardware.

Why it may be worth prioritising

This is the default path: any Arm CPU user running DeepSeek-V4-Flash hits an abort with no
indication that a flag would fix it. The model is widely downloaded at the moment, and the failure
looks like a broken model rather than a backend issue, because the weights load first and the crash
only arrives on the first decode.

Additional notes

Not related to quantized KV. -ctk/-ctv f16 throughout, because #26423 (quantized KV producing
garbage on deepseek4) is a separate open issue on this architecture.

x86 is unaffected: the same models and flags run without --no-repack on x86_64.