Merge llama.cpp v0.7.0 (b11669) by nbritton · Pull Request #22 · mxxm-t/mx-llama.cpp

nbritton · 2026-10-22T21:58:21Z

Merges the llama.cpp v0.7.0 release (620d222, b11669) into master. It is one merge commit on top of your master at c4d12e2c0, the same way you merged b11553. The branch fast-forwards from master, and the PR contains no other changes.

This replaces #17. Your b11553 merge made that PR obsolete. Unlike #17, this one does not change the RCCL default; GGML_HIP_RCCL stays OFF as in your tree.

What v0.7.0 brings

There are 116 upstream commits since b11553. The ones that touch code this fork changes:

cuda, hip: GDN chunked kernel ggml-org/llama.cpp#29353: an MMA-based chunked GDN kernel (NVIDIA Ampere GA10x, Ada and Blackwell; on HIP, RDNA3.5 only).
spec : add lilicorr support ggml-org/llama.cpp#30506: LiliCorr speculative drafting.
llama : do not map foreign ranges into a backend buffer ggml-org/llama.cpp#30405: model buffers map only the file ranges of their own context. llama_buf_map is gone.
llama : report compute graph reallocations during inference ggml-org/llama.cpp#30329: the scheduler is created in llama_context::sched_new() and reports graph reallocations.
meta: recreate all subgraphs when the arena is reallocated ggml-org/llama.cpp#30453: meta-backend subgraph recreation, which matches your existing fix.
ggml: Change default visibility of symbols to hidden ggml-org/llama.cpp#30014: hidden default symbol visibility.

How the merge was resolved

Every fork feature is kept. Where both sides changed the same code, the two changes are combined, not chosen between.

Textual conflicts (10 hunks in 8 files):

common/speculative.cpp: upstream's LiliCorr setup is placed ahead of the fork's fallback to the target vocab's mask token.
ggml/src/ggml-backend-meta.cpp: meta: recreate all subgraphs when the arena is reallocated ggml-org/llama.cpp#30453 is the fork's own fix, so the fork's comment is kept.
ggml/src/ggml-cuda/common.cuh: the fork's context fields are kept, with upstream's = {} initializers for the stream and cuBLAS arrays.
ggml/src/ggml-cuda/gated_delta_net.cu: both the fork's chunk kernel header and upstream's MMA header are included.
src/llama-context.cpp: both call sites use upstream's sched_new(), and the fork's sync-non-graph-inputs policy moves into it.
src/llama-ext.h: the fork's MTP prefill and pre-norm accumulation declarations are kept.
src/llama-model.cpp: the fork's skip for contexts with nothing to allocate is kept, without the removed per-file buffer map. load_tensors() ends with load_tensors_hadamard() and then the PLE prefault.
tools/server/server-context.cpp: release() resets the speculative state as before, and also drops the prompt of stateless tasks as upstream now does.

Two places merged without a textual conflict but would have been wrong, so the merge commit changes them:

LiliCorr with the deferred DFlash prompt encode. LiliCorr stores the projected rows of each inject in the eager path only. The fork's deferred encode replays injects in flush_pending_upto() without that step, so a LiliCorr drafter would draft from empty state. A LiliCorr drafter now stays on the eager path.
The new MMA GDN kernel with the snapshot ring. The MMA kernel writes only the final state, while the recurrent-snapshot ring needs states 1..K-1 too. When K > 1 the call goes to the fork's kernels, as before. On gfx906 the MMA path is never taken.

Testing

All tests ran on a Mac Pro 7,1 with 2x Radeon Pro Vega II Duo (4x gfx906, 32 GB each), ROCm 10.0.0-gfx906+20261018172427 (HIP 7.15.26333) and kernel 7.0. The build used the tree's own defaults (-DGGML_HIP=ON -DAMDGPU_TARGETS=gfx906, RCCL off), with GPUs capped at 125 W.

Build: 0 errors. All 7 warnings come from fork code this merge does not change: an unused variable in delta-net-base.cpp, an unused parameter in deepseek41.cpp, and five occupancy notes from the q8_repack GEMM.
test-backend-ops: every op passes on every die, 18644/18644 on each of the four.
Perplexity (Qwen3.8-27B Q8_0, wikitext-2, 16K x 6 chunks, -sm tensor): 5.6122 +/- 0.0623. Merge llama.cpp v0.5.0 (b11146) and build with RCCL by default #17 measured 5.6153 and the pre-v0.5.0 builds 5.6171.
-sm layer with Qwen3.8-27B: greedy generation is coherent.
MTP (Qwen3.8-27B, llama-server --spec-type draft-mtp --spec-draft-n-max 2, -sm tensor): 256 tokens generated, 153 of 203 drafts accepted (75%), 69.2 tok/s.
Qwen3.8-Flash-Next (qwen4exp) UD-Q4_K_XL (-sm tensor --n-cpu-moe 41 -lm mlock, LLAMA_PLE_SHARD=1): loads, and greedy generation is coherent.
-lm dio, about @xmesaj2's comment on Merge llama.cpp v0.5.0 (b11146) and build with RCCL by default #17: on this branch the model file is opened with O_DIRECT (checked with strace) and no read falls back to buffered I/O, on both the GPU and the CPU load path. I could not reproduce the fallback on any build, including Merge llama.cpp v0.5.0 (b11146) and build with RCCL by default #17's branch, a pre-v0.5.0 build, and your master c4d12e2c0 (built with our gfx906 patches on top, which do not touch the loader). The fallback warning names the errno (EINVAL/EFAULT from read(), or the open() error), and that would show what differs on the affected setup.

Not tested (no hardware or models here): CUDA and NVIDIA builds (which includes the new MMA GDN path), AMD architectures other than gfx906, DeepSeek-V4/V4.1, DSpark, DFlash, LiliCorr, gemma, MiniMax, and -tps pipeline stages.

🤖 Generated with Claude Code

…ng (ggml-org#29998) * vulkan: compute every row of an expert in mul_mm_id when ids repeat * vulkan: handle mul_mat_id duplicates in the prepass rather than looping Extend the "hoist row ids" optimization to always be enabled and to emit a compact list of tile descriptions that need to run, and to emit multiple tiles when needed. Then we can launch a tighter upper bound on the number of workgroups and the tail can trivially early exit. --------- Co-authored-by: Anjielon <[EMAIL_REDACTED]>

* Runtime support for Prism Bonsai 2 27B Assisted-by: Claude Code * address prism hadamard runtime feedback * move hadamard tensors into method, define folded weight * load_* hadamard fixes * cont : clean-up * conversion cleanup Assisted-by: Claude Code * prism hadamard key and methods for converter Assisted-by: Claude Code * address review bot feedback Assisted-by: Claude Code --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

…rg#29544) * rpc : turn GGML_RPC_DEBUG into a verbosity level and add logs GGML_RPC_DEBUG is now parsed as a number: 0/unset disables debug logs, 1-3 emit increasingly detailed output (events, per-command trace, transport detail). Non-numeric values fall back to 1. The duplicated env/macro blocks in ggml-rpc.cpp and transport.cpp are replaced by a shared log.h, which becomes the single choke point for all logging of the RPC backend: LOG_ERROR/LOG_WARN/LOG_INFO for unconditional severity logs and LOG_DBG/LOG_DBG2/LOG_DBG3 for the verbosity-gated ones. The transport files no longer need ggml-impl.h, and the server banner now also goes through the ggml logger (stderr). Missing logs are added on both the client (handshake, buffer ops, tensor transfers, graph computes, cache decisions) and the server (per-command dispatch, graph nodes), including the negotiated transport via the new socket_t::transport_name(). Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL * rpc : align log levels with the documented verbosity semantics The first pass introduced GGML_RPC_DEBUG levels, but many call sites did not follow the documented semantics and some logs bypassed the macros entirely: - Route the remaining raw GGML_LOG_* call sites through the local LOG_* macros so that log.h stays the single choke point for RPC logging - Emit per-tensor transfers and per-command handler traces at level 2, and one-shot lifecycle events (backend and buffer type creation, buffer allocation, graph compute, graph cache eviction) at level 1 - Drop the duplicated enqueue trace in rpc_dispatcher::send, work() already logs every dispatched command together with its round-trip timing - Promote degraded-operation events to LOG_WARN: remote allocation failure, RDMA falling back to TCP, unexpected peer disconnect - Log graph cache eviction on both sides, since client and server must stay in sync for the incremental graph update to be valid - Distinguish Unknown command (opcode out of range) from Unhandled command (valid opcode with no handler) - Fix format specifiers (%zu for size_t, %u for device ids, 0x for hex pointers) and a missing newline in an error log - Document that llama.cpp applications drop ggml debug records unless the verbosity threshold is raised with -lv 5 Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next

…s_norm (ggml-org#30310) * opencl: fallback when Q4_K weight images exceed device limits Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com> * opencl: avoid incomplete subgroup in rms_norm * opencl: refine conditions for dense bin kernels and fix bcast for q4_0 Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com> --------- Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>

…0266) * opencl: enable Gemma-4 E4B GPU decode Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com> * opencl: extend Gemma-4 GPU decode to E2B Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com> * opencl: optimize DK64 GQA8 decode Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com> * opencl: optimize DK128 GQA4 decode Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com> --------- Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>

* hexagon: add IM2COL_3D support Assisted-by: OpenCode * hexagon: add DMA paths for IM2COL_3D Assisted-by: OpenCode * hexagon: avoid scalar fp16 conversions in IM2COL Assisted-by: OpenCode * remove in-loop spills in pre-v81 devices Assisted-by: OpenCode * hex-im2col: re-write to use canonical dma dbl-buff pipeline, remove divs and very inefficient fp conversions * hex-im2col: move kernel_params and vtcm layout precompute into the host * hex-im2col: remove explicit tracking of number of dma-pushes * hex-im2col: remove incorrect uint64_t casts * hex-im2col: no need for macros for dma loops and pass compute func ptr via local context * hex-im2col: use cursor pattern, tracing and proper n_threads handling * hex-im2col: support non-contigue views * hex-im2col: minor reformatting --------- Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>

…ml-org#30326) Move the vk::Buffer stream operator to the shared header so it is visible to ggml-vulkan-debug.cpp.

See discussion in ggml-org#29997. This prevents a handful of internal symbols leaking via GGML_API.

…t block size (ggml-org#30164) * vulkan: specialize quant block size for CM2 matmul * vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size * vulkan: clarify CM2 quant block size comment

* ui: fix text attachment preview overflow Set the max height to 80dvh and the max width to full on screens below medium and 80vw from medium onward. * style: run formatter

…#30329) Add an optional ggml_backend_sched_realloc_callback that user code registers with ggml_backend_sched_set_realloc_callback to be notified when the buffers reserved for a graph are not sufficient and have to be reserved again. The callback reports whether the backend assignments and the graph size changed since the previous scheduling. llama_context registers the callback when creating the scheduler (wrapped into the new llama_context::sched_new helper) and treats the reallocations that follow a change of the backend assignments as expected, counting the rest by graph size - unchanged or changed from the previously scheduled graph. Both counters are reported on one line in the perf stats. Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next

* examples : fix draft sampling and rejection handling This commit fixes two issues in the speculative example which are described in detail in the linked issue. Resolves: ggml-org#29975 * examples : add kv-unified requirement for parallel > 1

* metal : fix mul_mat_id limits and kernel name * metal : fix mul_mat_id with duplicate ids kernel_mul_mm_id_map0 assumed at most one match per token and a fixed capacity of ne21 entries per expert, which produced wrong or dropped output rows and could overflow into the slice of the next expert when the ids contain duplicates. build a compact per-expert list of the (token, slot) entries instead: - pass 1 counts the entries of every expert, thread 0 computes the offsets of the expert slices (tpe now holds [base, count]) - pass 2 writes the entries with one unconditional store per token (conditional stores of divergent lanes are very slow), the dummy value of the tokens without a match lands on a padding slot of the slice that is never read - duplicates are handled by a rare branch - kernel_mul_mm_id reads the base/count of its expert and loops over the entry tiles that do not fit into the grid The ids buffer is sized as ne20*ne21 + ne02 (entries + padding). 883/883 test-backend-ops MUL_MAT_ID tests pass, llama-bench is within run-to-run variance of the previous behavior. Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD * metal : make mul_mat_id map0 ne02 a function constant use FOR_UNROLL for the id loops of kernel_mul_mm_id_map0 and turn ne02 into the function constant FC_mul_mm_id_map0_ne02 (FC_MUL_MM + 7), so that the threadgroup offsets of cnt/bas/sids and the prefix-sum bound fold at compile time. the pipeline name now encodes ne02 (same convention as the ssm_conv nc/silu constants) and the map0 pipeline is compiled with the constant value set. ne02 is no longer part of ggml_metal_kargs_mul_mm_id_map0 - the dispatch grid still comes from the tensor ne[2] on the host. no measurable perf change: 883/883 test-backend-ops MUL_MAT_ID tests pass, op-level and llama-bench results are within run variance. Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD * cont : clean-up + TODO * metal : dispatch mul_mat_id entry tiles from the prepass follow the approach of the vulkan implementation (ggml-org#29998): instead of launching a grid over experts x ne21 entry tiles and letting most of the threadgroups early exit, kernel_mul_mm_id_map0 now also emits a row map - the total tile count plus one (expert, first entry, valid rows) descriptor per 32-entry tile that has work. kernel_mul_mm_id is launched with a static upper bound on the number of tiles (full tiles over all entries plus at most one partial tile per expert), reads its descriptor and processes a single tile - the tail early exits. This removes the round loop that was needed to cover duplicate ids past the grid (register pressure: th_max 768 -> back to the plain mul_mm level) and duplicates are now handled implicitly, since they only produce more descriptors within the bound. The grid shrinks by ~8-16x on typical shapes (e.g. 128 experts, 8 used, 512 tokens: 128 tiles instead of 2048 entry tiles, of which 94% only launched to exit) and ne21 is no longer needed by the mm kernel kargs. 883/883 test-backend-ops MUL_MAT_ID tests pass. llama-bench (Qwen3.6-35B-A3B Q8_0) is now faster than master on all pp sizes (pp2048/4096/8192 +2%, pp512 +0.5%), tg32 unchanged. Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD * cont : add smem assert

…0299) Give llama_build() and llama_build_and_test() a LINK keyword (default: llama-common) so call sites can declare what a test actually needs: - ggml-only tests now link just ggml: test-opt, test-metal-graph-optimize, test-backend-ops, test-tiled-mulmat, test-barrier, test-quantize-fns, test-quantize-perf, test-rope, test-col2im-1d, test-gguf, test-alloc - llama-only tests drop llama-common: test-unicode, test-sampling, test-grammar-parser, test-llama-grammar, test-batch-alloc, test-gbnf-validator, test-quant-type-selection - both mtmd tests link mtmd only, dropping the LLAMA_TEST_NAME / target_link_libraries workarounds Both helpers now set cxx_std_17 explicitly, since the tests no longer inherit it from llama-common's PUBLIC interface. tests/testing.h no longer includes common.h; the only symbol it used from there (string_join) is inlined. Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next

* models: support pplx-decider * vllm d2 * update * clarify naming rule * stricter validation * fix ci * rm redundant filter_tensors

* models: support pplx-decider * vllm d2 * update * clarify naming rule * stricter validation * d3

) * vulkan: pack FMA for f16 accumulation, no dot product op * vulkan: dot_product packs two rows per f16 FMA * vulkan: rename dot_product to dot_product_v2

…30341) * hexagon: move the converting CPY in blocks of rows per DMA descriptor so short rows are not DMA-latency bound * hex-cpy: cleanup cpy cursor handling * hex-cpy-concat: introduce threaded dma/hvx gather-row helper Co-authored-by: Ruslan Karymov <67752306+karusrus@users.noreply.github.com> * hex-cpy: remove leftover rejections for extended buffers * hex-cpy: relax the gate for fast-path * hex-cpy: remove scalar divs and vtcm access and use explicit gather_sync * hex-cpy: add proper support for mdev work splitting to fastpath * hex-cpy: optimize dense gather-row tasks * hex-cpy: unroll gather-rows loops * hex-cpy: add fastpath for f16 * hex-cpy-concat: support for colapsing 3/4D dims * hex-cpy: double-buffer support * hex-cpy: factor out gather-rows into more of an op by itself * hex-cpy-concat: dedup kernel-params * hex-cpy-concat: step-down nr to fit instead of hard fail * hex-gather: replace div with a mask * hex-gather-rows: cleanup dma pipeline and remove the stalls * hex-gather: specialize compute funcs * hex-gather: reduce scalar reg.spills and func calls * hex-gather: do not recompute rows-per-thread on the npu * hex-build: add support for building/installing just hexagon * hex-cpy-concat: remove redundant barrier in concat and move cpy barrier under fence * hex-gather: annotations for dma-out * hex-mdev: make all key vtcm_acquire sync state atomic (fixes random hangs in mdev setups) --------- Co-authored-by: Yiwei Shao <yiwei@aizip.ai> Co-authored-by: Ruslan Karymov <67752306+karusrus@users.noreply.github.com>

…g#28748) * metal: fix metal tensor kernel_mul_mm error for large tensor Assisted-by: Codex * Update ggml/src/ggml-metal/kernels/mul_mm.metal Co-authored-by: YiChen Lv <63285796+forforever73@users.noreply.github.com> --------- Co-authored-by: 向锡铭 <xiangximing@xiangximings-MacBook-Air.local> Co-authored-by: YiChen Lv <63285796+forforever73@users.noreply.github.com>

…g#30416) The no-cache path builds a [n_tokens, n_tokens] mask and passes K and V unpadded, so FLASH_ATTN_EXT sees K->ne[1] == n_tokens. The CUDA kernels for head size 512 require a multiple of FATTN_KQ_STRIDE, so the op falls back to the CPU for most lengths, and the -fa auto probe (1 token) disables flash attention for such models. Pad the mask like llama_kv_cache::get_n_kv and pad K and V with zeros up to the KV dimension of the mask. clef builds its own causal mask and pads it the same way.

* model: fix vllm d3 multi seq case * add TODO

…iants (ggml-org#30324)

…org#30245) * tests: cover weighted scalar broadcasts Assisted-by: OpenCode * ggml : support extended scalar DMA binary ops Assisted-by: OpenCode * hex-binary: remove all non-dma paths * hex-binary: add support for non-contig strides and cleanup dma preable a bit * hex-binary: cleanup the gating and simplify dma retry * hex-geglu-erf: add a gate for F32 only, we dont support F16 for this op * tests: cover weighted repeat broadcast Assisted-by: OpenCode * hex-binary: update REPEAT and COMPLEX kernels to use dma as well (missed them last time) * hex-add-id: add dma support for src2 (idx) --------- Co-authored-by: Max Krasnyansky <maxk@qti.qualcomm.com>

…kernels (ggml-org#30198) * models: remove redundant ggml_cont calls now supported by CUDA unary kernels * models: restore cont_2d after sigmoid for broadcast mul, clean dflash cont * Update src/models/qwen3next.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * models: restore cont_2d before sigmoid in qwen3next --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

* ggml-zdnn: initial prototype of q8_0 matmul Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * docs: update zdnn docs Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> --------- Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

An embedding-only graph has no logits, and build_pooling() returns early when embeddings are off, so since the models no longer expand t_embd the graph ends up empty and its inputs are never allocated. Expand t_embd in that case.

* llama: honor PARTIAL_ONLY in the plain KV cache A non-SWA llama_kv_cache rolls back with seq_rm, so it holds no partial state, yet state_write ignored the flag and serialized every cell. A draft context made of a plain KV cache (MTP head, dense -md draft) therefore copied its whole KV into each server checkpoint, growing it by the draft KV size per token. The cache now writes and reads nothing for PARTIAL_ONLY unless it is SWA, and the iSWA wrapper no longer gates its base cache since it filters itself. * Update src/llama-kv-cache.cpp Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * llama: delegate PARTIAL_ONLY to the caches of every memory wrapper The hybrid, hybrid-idx and dsa-iswa wrappers skipped their attention caches on PARTIAL_ONLY, which also dropped an SWA window that is partial state. Each cache now applies the flag itself, so the wrappers only delegate. The indexer is marked stale only when a restore rewrites cells. test-llama-context gains a case checking that the partial state is not larger than the full state and, without SWA, does not grow with the tokens. --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

…ml-org#30491) * metal : add BF16 variant of the tensor API flash attention kernel * cont : move tensor FA kernels to kernels/fa_tensor.metal

…-org#29200) * cpu: add F16 SWIGLU_OAI reference implementation and F16 activation op test coverage Adds ggml_compute_forward_swiglu_oai_f16() to the CPU backend, mirroring the existing F32 path, and extends test-backend-ops with F16 test cases for SWIGLU_OAI so downstream backends(e.g. Hexagon) can be CPU-diffed for correctness. * ggml-openvino: fix f16 SWIGLU_OAI type mismatch in alpha/one constants translate_glu_swiglu_oai creates alpha_const and one as f32 constants unconditionally, so f16 inputs hit a type mismatch in Swish/Add on GPU ("Swish inputs must have the same type but they are: f16 and f32"). translate_glu_swiglu_clamp already upcasts to f32 and converts back at the end; apply the same pattern here. --------- Co-authored-by: ravi9 <ravi.panchumarthy@intel.com>

…y channels (ggml-org#27703) * vulkan : copy strided f16 KV once so reads spread across memory channels Assisted-by: Claude (Opus 4.8 / Opus 5) * tests : cover the f16 KV copy to scratch Assisted-by: Claude (Opus 4.8 / Opus 5) * vulkan : gate the f16 KV scratch copy to AMD Assisted-by: Claude (Opus 4.8 / Opus 5) * vulkan : reuse the generic copy shader for the f16 KV scratch fill The f16 clause needs only a strided copy, not a dequant, so drop the dedicated shader and dispatch the existing copy pipeline instead. The copy is described in source memory order so the strided access lands on the writes, which stream, instead of the reads, which would concentrate on a few memory channels. Assisted-by: Claude (Opus 4.8 / Opus 5 / Fable 5) * vulkan : extend the strided KV scratch copy to bf16 Assisted-by: Claude Opus 5 * vulkan : clean up comments Assisted-by: Claude Opus 5 * consolidate copy shaders * disable on RDNA4 --------- Co-authored-by: Ruben Ortlam <rortlam@redhat.com>

…g#30532) * llama : split llama_prefetch() into merge and prefetch steps Merge the ranges first, then call the system functions. This fixes -Wunreachable-code-return on Windows. Signed-off-by: Adrien Gallouët <angt@huggingface.co> * Use const_cast Signed-off-by: Adrien Gallouët <angt@huggingface.co> --------- Signed-off-by: Adrien Gallouët <angt@huggingface.co>

…#30485) * unicode : reject overlong UTF-8 and out-of-range codepoints * Update tests/test-unicode.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * tests: document byte-wise UTF-8 error recovery * break out of first test on failure --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

Assisted-by: Codex

…rg#30421) Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>

* replace oneAPI package by DLE+oneDNN package * support to install oneDNN in windows, in release and docker * for internal test * remove test code * Update intel.Dockerfile

* vulkan : fix concurrent first device init * tests : load models in parallel in test-thread-safety Loading them serially never exercised concurrent first backend init, so the crash reported in ggml-org#29764 was invisible to this test. * vulkan : create fence before retaining device in buffer context

tojson wrote nan, inf and -inf for these floats, as values and as dict keys; json.dumps writes NaN, Infinity and -Infinity. The string and safe filters of a float keep nan and inf, as Python's str() does, so they no longer use the tojson function. float_to_json is renamed float_repr, as it now gives str(); float_json gives the json.dumps text. Also say in a comment why a last digit 9 is not bumped at a power of 2.

Assisted-by: Codex

* Add Support to maion-coder arch * iAdd Support to support Draft models * Address comments * Remove maion context shift function --------- Co-authored-by: Karthik Vijayan <kavijay@microsoft.com>

…gml-org#30582) The registry hard-coded 10 for AGENTIC_MAX_TURNS while DEFAULT_AGENTIC_CONFIG said 100. The registry seeds SETTING_CONFIG_DEFAULT, so the loop's `|| DEFAULT_AGENTIC_CONFIG.maxTurns` fallback never fires and users got 10 turns. Reference the shared constant, as the adjacent MCP timeout does. Assisted by: llama-ui (ggml-org/Qwen3.8-Flash-Next:IQ4_NL)

* bump ty to 0.0.88 * arange -> linspace.tolist() to pass expected type

…ggml-org#30567) Slot selection takes free slots by id or by last use only, so child tasks often land on a slot holding a parent state while empty slots are idle, and overwrite it without saving it to the prompt cache. Free slots are now ordered empty first, then least recently used, for both the LRU fallback and the child slots, and a child task saves the state it overwrites. With 4 grouped decision requests in flight and 12 states in rotation, prompt cache reads go from 5 to 36. [no release]

The slot file held the target context and the context checkpoints, but never the draft context, so a restored slot drafted with an empty or stale draft KV until the cache refilled. The file now ends with the live draft state, the full draft context and the speculative state, in the same appendix style as the checkpoints. A draft state that does not fit the draft context is dropped and the draft starts empty, while older files and older servers keep working. [no release]

…rg#30578) * model : mellum - read MoE gating params from GGUF, unify RoPE - read/write expert_weights_norm and expert_gating_func instead of hardcoded values (defaults unchanged) - use per-layer rope params via get_rope_freq_base/scale - check n_expert once, outside the layer loop * Update conversion/mellum.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> [no release]

The first evaluated token attends to the positions above pos - n_swa, so a SWA state starting at pos - n_swa + 1 holds the whole window, but the reuse threshold asked for two more cells. A saved SWA state keeps only the cells inside its window, so a state read back from a slot file or from the RAM prompt cache was always rejected and the slot rolled back to a checkpoint. The SWA threshold now matches the window, while a state without SWA keeps the exact bound. The test harness gains override_kv, used to give the SWA test model a window smaller than the test context. [no release]

Small helper around the GitHub Actions API: lists the workflow runs that are not finished yet (in_progress, queued, pending, waiting, requested), grouped by status. Supports selecting runs by status, by branch (server-side, to reduce API traffic) or by a regex on the branch name, and can cancel the selected runs via --kill. Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next

…gml-org#30520) * add support * remove unneeded bos fix * add video flag * fix flake error * remove video-specific budgeting * convert : fix lilicorr-dflash bid clash * spec : enforce lilicorr n_tokens condition * spec : fix lilicorr walk [no release]

Brings the fork from upstream b11553 to the v0.7.0 release (116 upstream commits). Every fork feature is kept; where both sides changed the same code the two changes are combined, not chosen between. Textual conflicts (10 hunks, 8 files): - common/speculative.cpp: upstream's LiliCorr setup (ggml-org#30506) placed ahead of the fork's fallback to the target vocab's mask token. - ggml/src/ggml-backend-meta.cpp: upstream's subgraph recreation (ggml-org#30453) is the fork's own fix; the fork's comment is kept. - ggml/src/ggml-cuda/common.cuh: the fork's context fields kept, with upstream's `= {}` initializers for the stream and cuBLAS arrays. - ggml/src/ggml-cuda/gated_delta_net.cu: includes both the fork's chunk kernel header and upstream's MMA header. - src/llama-context.cpp: both call sites use upstream's sched_new() member (it adds the graph realloc callback, ggml-org#30329); the fork's sync-non-graph-inputs policy moves into it. - src/llama-ext.h: the fork's MTP prefill and pre-norm accumulation declarations kept, upstream's wording for the decision head comment. - src/llama-model.cpp: the fork's skip for contexts with nothing to allocate kept without the per-file buffer map that upstream removed (ggml-org#30405); load_tensors() ends with upstream's load_tensors_hadamard() and then the fork's PLE prefault. - tools/server/server-context.cpp: release() resets the speculative state as the fork does and also drops the prompt of stateless tasks as upstream does. Merged without a textual conflict but changed here: - common/speculative.cpp: LiliCorr stores the projected rows of each inject in the eager path only. The fork's deferred prompt encode replays the injects in flush_pending_upto() without that step, so a LiliCorr drafter stays on the eager path. - ggml/src/ggml-cuda/gated_delta_net.cu: upstream's MMA chunked kernel (ggml-org#29353) writes only the final state. When the fork's snapshot ring keeps more than one state (K > 1) the call goes to the fork's kernels. On HIP the MMA path runs on RDNA3.5 only, so gfx906 does not take it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

PR mxxm-t#17 was closed when mxxm-t merged b11553; the Exabit commits moved onto that master and then onto merge-v0.7.0. The release procedure is now a rebase of the Exabit commits onto each merge-vX branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…cycles - RELEASE-RUNBOOK.md: the procedure for the next upstream release (reconcile with mxxm-t, merge, silent-break checks, rebase, build and test, push and PR, packages, publishing, dogfooding) and the traps found on v0.7.0. - reports/2026-10-22-v070-rebase.md + data/raw/2026-10-22/: the cycle's measurements (suite on merge, master and the installed deb; fold attribution and KLD; dio; packages and the PyTorch stack check). - tools/release/: the scripts of the cycle; tools/gpu-test-env.sh: the macpro-fand version the box already runs. - CLAUDE.md, README.md (Status 2026-10-22), REQUIREMENTS.md change log, RE-RE-SURVEY-ACTION-PLAN.md, ROCM-SETUP.md, reports/README.md: substrate merge-v0.7.0 (PR mxxm-t/mx-llama.cpp#22), master ef70ef51e, the installed debs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

jeffbolznv and others added 30 commits October 11, 2026 14:02

cpu: fix AVX2 CPUs not using F16C when available (ggml-org#30270)

1cf7767

hexagon: optimize q2_k bpw and activation (ggml-org#30035)

e937b17

vulkan: fix buffer logging on 32-bit platforms after source split (gg…

b2f036c

…ml-org#30326) Move the vk::Buffer stream operator to the shared header so it is visible to ggml-vulkan-debug.cpp.

ggml: Change default visibility of symbols to hidden (ggml-org#30014)

a05e58c

See discussion in ggml-org#29997. This prevents a handful of internal symbols leaking via GGML_API.

sycl: guard DMMV reads at row tails (ggml-org#29338)

f9a95ae

ui: fix text attachment preview overflow (ggml-org#29898)

2f160ba

* ui: fix text attachment preview overflow Set the max height to 80dvh and the max width to full on screens below medium and 80vw from medium onward. * style: run formatter

model: support vllm decision 2 model (ggml-org#30378)

097cf33

* models: support pplx-decider * vllm d2 * update * clarify naming rule * stricter validation * fix ci * rm redundant filter_tensors

model: support vllm d3 models (ggml-org#30383)

efe3626

* models: support pplx-decider * vllm d2 * update * clarify naming rule * stricter validation * d3

vulkan: pack FMA for f16 accumulation, no dot product op (ggml-org#29877

0f701c8

) * vulkan: pack FMA for f16 accumulation, no dot product op * vulkan: dot_product packs two rows per f16 FMA * vulkan: rename dot_product to dot_product_v2

model: fix vllm d3 multi seq case (ggml-org#30408)

cbe28bb

* model: fix vllm d3 multi seq case * add TODO

ci: fix OpenVINO download links (ggml-org#30401)

a49f58d

ci : build OpenVINO releases with GGML_NATIVE=OFF and CPU backend var…

8f574b0

…iants (ggml-org#30324)

ggml-zdnn : introduce Q8_0 mat mul (ggml-org#29836)

af50f2f

* ggml-zdnn: initial prototype of q8_0 matmul Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> * docs: update zdnn docs Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> --------- Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

graph : avoid reallocations in the "no cache" path (ggml-org#30439)

9880157

ServeurpersoCom and others added 28 commits October 15, 2026 06:38

metal : add BF16 variant of the tensor API flash attention kernel (gg…

6e66cb5

…ml-org#30491) * metal : add BF16 variant of the tensor API flash attention kernel * cont : move tensor FA kernels to kernels/fa_tensor.metal

opencl: apply flash attn softcap before mask (ggml-org#30546)

77c1f39

ggml-cpu : optimize pad_f32 (ggml-org#30526)

23f83e0

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

jinja : fix loop body variable scoping (ggml-org#30533)

2130ca8

Assisted-by: Codex

opencl: fix OOB tail access for adreno noshuffle gemv kernels (ggml-o…

9166cea

…rg#30421) Co-authored-by: Hongqiang Wang <wangh@qti.qualcomm.com>

sycl : replace oneAPI package by DLE+oneDNN package (ggml-org#30131)

e36df60

* replace oneAPI package by DLE+oneDNN package * support to install oneDNN in windows, in release and docker * for internal test * remove test code * Update intel.Dockerfile

jinja : improve context copy constructor performance (ggml-org#30562)

08698c8

Assisted-by: Codex

models : add maion-coder (ggml-org#29778)

3e4dfa8

* Add Support to maion-coder arch * iAdd Support to support Draft models * Address comments * Remove maion context shift function --------- Co-authored-by: Karthik Vijayan <kavijay@microsoft.com>

ci : bump ty to 0.0.88 (ggml-org#30587)

711780f

* bump ty to 0.0.88 * arange -> linspace.tolist() to pass expected type

ggml : bump version to 0.27.0

4ed1c85

sync : ggml

c0cb53a

llama.cpp : bump version to 0.7.0 (ggml-org#30603)

620d222

Navigation Menu

Repository navigation

Merge llama.cpp v0.7.0 (b11669) - #22

Merge llama.cpp v0.7.0 (b11669)#22
nbritton wants to merge 117 commits into
mxxm-t:masterfrom
exabit-io:merge-v0.7.0

nbritton commented Oct 22, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

nbritton commented Oct 22, 2026

What v0.7.0 brings

How the merge was resolved

Testing

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants