Skip to content

Move llama.cpp to v0.9.0 and audit its CPU ISA source pair - #175

Merged
leehack merged 4 commits into
mainfrom
chore/llama-cpp-v0.9.0
Nov 7, 2026
Merged

leehack merged 4 commits into
mainfrom
chore/llama-cpp-v0.9.0

Conversation

@leehack

@leehack leehack commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

Moves third_party/llama.cpp from upstream v0.8.0 (8642541b) to stable v0.9.0 (cd959bae, 2026-11-04, v0.8.0 + 150 commits) and audits its CPU ISA source pair.

Head ef32565, based on main f3961b7 (merged in, no rebase).

Merging this publishes nothing. Since #178, auto_native_release.yml runs only on manual dispatch. The 2026-11-05 automatic dispatch failed only at the Linux arm64 and Android arm64 CPU ISA audits (Unaudited ggml/src + kai source fingerprint pair ('fead7601…', 'fd5b9ca9…')) and published nothing. A maintainer dispatches v0.9.0 after this lands. No tag, release or dispatch was made from this branch.

Changes

  • Submodule: v0.8.0 → v0.9.0 (separate commit).
  • validate_wrapper.yml: "v0.9.0": "cd959bae…" added to LLAMADART_UPSTREAMS. It is the pin row, so it builds the checkout as is and runs the pinned-only steps; tools/ci_scope.py resolves upstreams = [v0.7.0, v0.8.0, v0.9.0], pinned = v0.9.0. The v0.7.0 and v0.8.0 rows stay (maintainer decision).
  • Android/Linux arm64 ISA: the v0.9.0 ggml/src digest bound to the unchanged KleidiAI v1.32.0 kai digest, granting exactly the v0.8.0 function set.
  • Wrapper: no source change. mtmd_gen_audio_info.n_voices and the removed mtmd_helper_gen_audio_inp.voice are not read; Chatterbox reports UNKNOWN/unsupported. Chatterbox and predefined voices are out of scope.
  • Docs: docs/v090_qualification.md (wrapper, per-patch review, consumer ABI delta), docs/v090_android_isa_qualification.md (source pair, FHM re-check with 0005/0006), AGENTS.md pointers. docs/native174_f16_dot_accumulation.md no longer says a daily detector publishes v0.9.0.

Merge of main (#177, #178)

File Resolution
.github/workflows/validate_wrapper.yml Conflict. Took main's LLAMADART_UPSTREAMS form; added the v0.9.0 row. The old per-job v0.9.0/pinned matrix edits are gone with that form.
tests/test_ci_scope.py Conflict. Took main's version; it already checks that the pin is exactly one row and that the v0.7.0/v0.8.0 rows remain.
tools/validate_android_cpu_isa.py, tests/test_android_cpu_isa.py, AGENTS.md, docs/platform_backend_strategy.md Auto-merged: #177's +fp16fml decoding and test sit next to this PR's v0.9.0 pair.

Carried patches

Reviewed 2026-11-07 against v0.9.0 and upstream master f7591a6 (b12190). apply --strict succeeds on both and applies 0001, 0002, 0005 and 0006, the four that manifest --upstream lists. Every patch passes git apply --check on both. series.json is unchanged.

Patch v0.9.0 Upstream Status
0001 Adreno 750 matvec Applied #29165 open draft; use_subgroups unchanged on master Keep
0002 buffer device address Applied No fix on master; still no entry-point check Keep
0003 embedding-mode reserve Not applied (upstream_fix, #31086) Fixed since v0.8.0 v0.7.0 row only
0004 Metal unary cast Not applied (upstream_fix, required_text absent, #31360) Fixed after v0.8.0 v0.8.0 row only
0005 ARM F16 dot Applied #31734 open; NEON still accumulates in float16x8_t Keep
0006 ARM F16 GEMM Applied Same; llamafile still tinyBLAS<8, float16x8_t, …> Keep

Measured: v0.7.0 manifest lists 0001–0006 but strict apply rejects 0004 (required_text absent), so it builds 0001–0003, 0005, 0006; v0.8.0 lists 0001, 0002, 0004–0006. Deleting 0003/0004 means dropping the v0.7.0/v0.8.0 rows — a maintainer support-policy call.

ISA audit

Source review unchanged from v0.8.0 except three ggml SVE256 paths in arch/arm/repack.cpp, behind __ARM_FEATURE_SVE and a runtime 256-bit check, so not in the non-SVE variants. The source pair still fingerprints unpatched upstream (fead7601… / fd5b9ca9… recomputed on the merged tree), so 0005/0006 do not change it.

Re-check on the merged tree (0005/0006 applied):

Module Audit FMLAL/FMLAL2
Android armv8.2_2 (NDK 28.2) PASS, 266,016 instr; 3,130 SVE/SME in the 22 functions 0
Android armv8.6_1 (NDK 28.2) PASS, 269,285 instr; 3,130 SVE/SME in the 22 functions 10, all in ggml_vec_dot_f16
Linux baseline / linux_armv8.2_2 / linux_armv8.6_1 (ubuntu:24.04 arm64, GCC 13.3, check_linux_arm64_cpu_variants.sh) PASS all three; QEMU selection correct on cortex-a53, cortex-a72, neoverse-n1, neoverse-v1, neoverse-n2; finite F16 dot check passes on all 5 0 / 0 / 10 (fhm=10/0)

validate_linux_arm64_cpu_isa.py allows fhm only with FP16 on ARMv8.4+, i.e. linux_armv8.6_1, not armv8.2_2. validate_android_cpu_isa.py only enforces SVE/SME containment (decodes FHM via +fp16fml) and CI/release run it on android_armv8.2_2 only; applying the Linux family policy to both Android modules also passes, with fhm allowed only for armv8.6_1.

Local evidence (merged tree)

Check Result
ci_scope.py with the workflow table upstreams [v0.7.0, v0.8.0, v0.9.0], pinned v0.9.0
macOS arm64 macos-arm64-full + tests (M4 Max) ctest 85/85 incl. llamadart_f16_dot_accumulation_test; validate_exports.py --forbid-import __cxa_atexit 92 exports; check-apple pass
Python test_*.py / *_test.py / verify_release_provenance.py 446 OK (1 skip) / 23 OK / pass
actionlint clean except the known windows-11-vs2026-arm label

Hosted evidence (head ef32565)

All 52 checks pass. Rows are now v0.7.0, v0.8.0 and v0.9.0 (#178 dropped the older rows). The v0.9.0 rows skip "Select candidate upstream" and run the pinned-only steps (Android bundle exception runtime, Linux static-destructor check), confirming it is the pin row. Includes the v0.9.0 Android ISA, emulated KleidiAI dispatch, Windows arm64 KleidiAI/Vulkan, Linux arm64 artifact contract (variant audit + QEMU), wrapper contract, macOS exit teardown (Release with the F16 conversion-path test, and ASan), and the Metal OS floor at the pin.

Not covered

  • No lane runs the new ggml SVE256 repack paths (they need an SVE variant on 256-bit SVE).
  • Real-model TTS smoke, and model, family and device qualification, are left to the consumer release.

Follow-up (not in this PR)

  • tools/validate_android_cpu_isa.py still allows the v0.4.0–v0.6.0 source pairs although no LLAMADART_UPSTREAMS row builds them any more.

Before merge

  • The 0.18.0 hold is cleared: llamadart v0.18.0 was published 2026-11-07 (leehack/llamadart#1746 merged).
  • Independent adversarial audit: ACCEPT at exact head ef32565 on main f3961b7f, no blocking findings: submodule bump and merge resolution, per-patch table, v0.9.0 source pair recomputed (fail-closed; mutations of the pair, function set and upstream row all fail tests), Android ISA rebuilt (FMLAL only in armv8.6_1), exports 92, check-apple, CTest 85/85. Wording nits (AGENTS.md calls 0004 "retired"; patches README line 73 mentions the automatic release) go into the v0.4.0–v0.6.0 source-pair prune follow-up.
  • Fresh CI if the head moves.
  • After merge: manually dispatch the stable release for v0.9.0, check its assets, then land #176, then the llamadart pin and bindings.

Add v0.9.0 rows to every validate_wrapper.yml lane and admit the reviewed
v0.9.0 ggml/src + KleidiAI v1.32.0 pair with the v0.8.0 function set.
Document the wrapper, carried-patch and consumer ABI delta.
Conflicts in validate_wrapper.yml and test_ci_scope.py take main's
LLAMADART_UPSTREAMS form; the v0.9.0 pin is added as its third row.
Patches 0001, 0002, 0005 and 0006 apply on v0.9.0 and current upstream
master; 0003 and 0004 stay only for the v0.7.0 and v0.8.0 rows. Merging
publishes nothing now that the stable detector is manual-only.
@leehack
leehack marked this pull request as ready for review November 7, 2026 16:06
@leehack
leehack merged commit b1ce76c into main Nov 7, 2026
52 checks passed
leehack added a commit that referenced this pull request Nov 7, 2026
#175 was squash-merged as b1ce76c; conflicts take main's version of its
files and keep this branch's llamadart_tts_voice_smoke build target.
@leehack leehack mentioned this pull request Nov 7, 2026
13 of 15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant