Skip to content

[0.17.1] Qualify llama.cpp v0.8.0 as the next runtime - #131

Merged
leehack merged 6 commits into
mainfrom
upgrade/llama-cpp-v0.8.0
Oct 29, 2026
Merged

leehack merged 6 commits into
mainfrom
upgrade/llama-cpp-v0.8.0

Conversation

@leehack

@leehack leehack commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Qualifies upstream stable v0.8.0 (8642541b, b11912 + 48 commits) as the next native runtime, for llamadart 0.17.1 (#112, plan llamadart#922). Supersedes #125 (b11912): its four commits are carried unchanged onto current main, plus one retarget commit.

Draft, not for merge. No tag, release, workflow dispatch or downstream pin change. The production submodule stays at v0.7.0.

Merging this publishes v0.8.0. The scheduled stable dispatcher already selects v0.8.0. Its 2026-10-29 dispatch from main failed in every build job because main's wrapper does not compile against the v0.8.0 mtmd API, so nothing was published. Once this is on main, the next scheduled run builds and publishes v0.8.0 as latest with no further code change. Merge only when the maintainer approves publication.

Changes

Carried patches

All three apply strictly (llama_cpp_patches.py apply --strict).

Local evidence (macOS arm64, M4 Max)

Check Result
Wrapper contract build (Debug, CI flags) + ctest, v0.8.0 with patches 57/57
#114 patchless build (Release, CPU, Metal/KleidiAI off, 0003 removed) + ctest 57/57
validate_exports.py / grammar boundary 89 exports / 2000 accepted, 2001 rejected
Android armv8.2 ISA (NDK 28.2) PASS: 261,765 instructions; 3,130 scalable, all in 22 exact functions
KleidiAI dispatch test, SME on and off 128 masks + 12 matmuls pass
Python suite 279 OK, 1 skip; *_test.py 22 OK
actionlint clean except the repo's known windows-11-vs2026-arm label
Qwen3-TTS 1.7B Q4_K_M + Q8_0 projector tts_smoke (CPU, model hashes as in #125) PASS: same seed repeats PCM, distinct seeds differ, cancel checks pass

Not covered

Hosted evidence (head defccfb)

  • Release provenance: pass.
  • Native validation: 60/64 jobs pass, including every v0.8.0 row except Linux arm64: Android ISA, Android Vulkan shaders, emulated KleidiAI dispatch, Windows arm64 KleidiAI and Vulkan, Windows x64 runtime closure, MSVC mtmd link, Linux x64 artifact contract, wrapper contract, macOS exit teardown (release and ASan).
  • Failing: linux-artifact-contract (arm64) on the pinned v0.7.0, v0.7.0 and v0.8.0 rows, plus the wrapper-validation aggregate. The Linux arm64 CPU ISA audit rejects four SVE instructions outside any sized function in the baseline libggml-cpu.so. The job now runs on ubuntu-26.04 / aarch64 GCC 15.2; the last green main run was on 24.04. It fails on the pinned row too, so this PR did not cause it: ubuntu-latest moved to 26.04: Linux CUDA release build and Linux arm64 ISA audit fail #132.

Add b11912 (961c376a) rows to every native validation lane and skip the
Qwen3-TTS projector RNG priming where mtmd seeds a fresh generation state
per synthesis (mtmd_gen_state, upstream #30902).
b11912 adds two SVE Q8_0 kernels reachable only behind runtime SVE256,
SVE I8MM and DOTPROD (mask 69). Bind them to the exact b11912 source pair
and extend the dispatch selector sweep to the new feature bit.
v0.8.0 (8642541b) is b11912 plus 48 commits. Every native lane's
candidate row now builds it instead of b11912.

The Android ISA policy binds the v0.8.0 ggml/src fingerprint to the
unchanged KleidiAI v1.32.0 source and its existing function set: the
only ggml-cpu change since b11912 is the non-contiguous scale op
(#31146). The armv8.2 audit passes with 3,130 scalable instructions in
the same 22 functions.

The three carried patches apply strictly. 0003 is redundant on v0.8.0
but stays while the pinned v0.7.0 builds from the same series.
@leehack

leehack commented Oct 29, 2026

Copy link
Copy Markdown
Owner Author

Independent audit at exact head 16f8a8d against base df1efa1: no P1 or P2 findings, and 64/64 checks pass, including every v0.8.0 row. Evidence is in llamadart-evidence/native-pr-131/.

Corrections to this PR's claims, none affecting code:

  • 89 is the unchanged count of required wrapper exports, not the number of new exports.
  • Some upstream struct layouts change: mtmd_helper_init_opt grows from 24 to 32 bytes, and mtmd_gen_* change. The core repin must account for both.
  • Patch 0003 still ships in v0.8.0, so removing it is a follow-up.

The v0.8.0 CUDA, HIP and Apple release rows have no proof yet. The release fails closed: package-and-release needs every build. Merging with maintainer approval.

@leehack
leehack marked this pull request as ready for review October 29, 2026 22:09
@leehack
leehack merged commit 5092f19 into main Oct 29, 2026
64 checks passed
@leehack
leehack deleted the upgrade/llama-cpp-v0.8.0 branch November 7, 2026 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant