Repository navigation
perf(mac): Apple CPU path trails llama.cpp — 0.53× prefill / 0.72× decode on bare-metal M1 Max #456
Description
First run of the ratio-gated Apple guard (#455, run https://github.com/geisten/geistlib/actions/runs/36195809559), gemma4 E2B Q4_K_M, pp512/tg64, 10 repeats, 3 threads on both engines, same GGUF, CPU-only (llama.cpp 2d8d612e, GGML_METAL=OFF, Accelerate default; geist TARGET=mac-omp, clang, cpu_neon + Accelerate):
| tok/s | prefill | decode |
|---|---|---|
| geist | 26.4 | 9.3 (±8 %) |
| llama.cpp | 66.8 | 18.8 |
| ratio | 0.40× | 0.50× |
benchmark/README.md claims geist leads prefill on Apple silicon at every length (1.48× at 1024 tokens on M1 Max). On the hosted runner it is at 40 %. Candidates, none verified: Accelerate/AMX not available or not used inside the runner VM (geist's fp32 path leans on it, llama.cpp's Q4_K path does not); thread placement on 3 vCPUs (geist auto-selects prefill/decode thread counts for physical cores); OMP_WAIT_POLICY=active on an oversubscribed VM. The three earlier red weeks (23.9 / 14.4 / 38.0 tok/s prefill) are consistent with this, not with runner noise alone.
The gate's floors are set to 0.25 / 0.3 until this is understood; raise them once the ratio moves. Reproduce locally on a quiet Mac first (tools/bench_mac_ab.py has the load gate) to separate "VM" from "code".
Activity
ci(mac): ratio floors under what the runner measures today (0.40x / 0…
geisten commentedon Sep 26, 2026
Resolved as "not a runner problem". Two pieces of evidence:
Probe on the hosted runner (temporary workflow, run https://github.com/geisten/geistlib/actions/runs/36258240829,
Apple M1 (Virtual), 3 vCPUs, no I8MM/SME, DotProd yes), gemma4 E2B Q4_K_M, pp512 mean of 3:* geist's region hooks pin prefill to
hw.perflevel0.physicalcpu(3) regardless ofOMP_NUM_THREADS, so the three geist columns are the same measurement; the VM has no AMX-relevant effect on geist because its Q4_K prefill never calls sgemm (native and Accelerate builds are within noise). tg64: geist 17-19, llama.cpp 26.The repository's own bare-metal cell,
benchmark/results/CROSS-ENGINE-APPLE-M1MAX.md(2026-09-03, M1 Max, 8 P-cores, same llama.cpp pin2d8d612e, 4 alternating A/B cycles, MAD ≤ 1.4 %): geist 0.53× prefill at pp512 (161 vs 302), 0.72× decode. The gap widens with depth.So the weekly guard's 0.37× / 0.73× is the real state of the Apple CPU path against current llama.cpp, plus some VM overhead on prefill (3 slow vCPUs where llama.cpp's BLAS path scales better). The three red Sundays before the ratio gate (#439) were this gap crossing an absolute floor that had been set from older llama.cpp-relative claims.
What is stale:
benchmark/README.md's TL;DR still says geist "wins prefill at every length" on Apple silicon, 1.48× at 1024 tokens — that was against llama.cppd05fe1dunder best-of-10 rules and is contradicted by the strict-protocol cell above (which the file's own historical-numbers caveat already hedges). The open perf question is the existing one: llama.cpp's Accelerate-BLAS prompt path beats geist's int8 W4A8 NEON kernels on Apple;benchmark/results/APPLE.mdanddocs/TUNING.mdare where that work is tracked.Guard floors (
apple-perf.yml, 0.25 / 0.3) stay under the bare-metal ratio with VM margin; raise them if the Apple CPU prefill path improves. The probe workflow branch is deleted.[-]perf(mac): geist at 0.40x prefill / 0.50x decode of llama.cpp on the hosted macos-15 runner[/-][+]perf(mac): Apple CPU path trails llama.cpp — 0.53× prefill / 0.72× decode on bare-metal M1 Max[/+]geisten commentedon Oct 1, 2026
Retitled: the runner question is answered (see the comment above), so this issue now tracks the remaining gap itself.
.github/workflows/apple-perf.ymlrefers to #456 for raising the guard floors (0.25 / 0.3), and that reference stays valid. Close this issue when the Apple CPU W4A8 NEON prefill path closes the gap to llama.cpp's Accelerate-BLAS path on the bare-metal cell (benchmark/results/CROSS-ENGINE-APPLE-M1MAX.md), and raise the floors then.Generated by Claude Code
geisten commentedon Oct 4, 2026
Status from the ticket sweep: nothing here can be measured outside the bare-metal M1 Max cell, so I'm leaving it for now. The only part doable without a Mac is the stale TL;DR claim in
benchmark/README.md(noted on 2026-09-26).Question: should I fix that doc line separately, or leave it until the floors are raised?
Generated by Claude Code
geisten commentedon Oct 8, 2026
Status (sweep 2026-10-08): still valid. This is Apple-only (Metal / Apple CPU path), and there is no Apple host or runner in this sweep; nothing on main changed this code path since the last status. Labelled
needs-hardware.Generated by Claude Code
cpu_neon: a macOS guest takes the Linux kernel set; Apple decode keep…
changelog: cpu_neon kernel set and decode threads in a macOS VM (#456)
Merge pull request #732 from geisten/perf/456-apple-cpu