[geisten]

Summary

Accelerate brings its own thread pool to every cblas call. Where geist has
already spread a matmul across its OpenMP threads — the quantized prefill
kernels dequantize a tile per thread and call sgemm on it — each of those
calls asks for a pool of its own, and the machine waits on itself. On a 27B
PQ2_0 prefill, sample(1) counted ~35 k samples parked in worker threads and
~11.6 k in OpenMP waits against ~30 k actually inside BLAS. That was the
remaining candidate for the CPU prefill gap to the PrismML fork measured in
#429 (13.6 vs 17.35 t/s at pp256).

The OpenBLAS branch of geist_blas_pin_single_thread has had the equivalent
pin since it was written; the Apple branch was an empty stub. This fills it in.

Changes

• src/backends/common/geist_gemm.c: an __APPLE__ branch of
  geist_blas_pin_single_thread that sets Accelerate's threading to match the
  caller — single-threaded inside an OpenMP region, Accelerate's own threading
  outside one.
• BLASSetThreading's setting is thread-local (vecLib thread_api.h), so
  it is set on the calling thread and only there. A lone caller — the small F32
  projections that run outside any parallel region — keeps Accelerate's
  threading rather than being pinned to one core. The value is cached per thread
  and rewritten only when omp_in_parallel() changes, so the steady state costs
  one comparison.
• Resolved through dlsym, so systems without it keep today's behaviour;
  GEIST_BLAS_THREAD_MATCH=0 disables it for A/B.

Testing

• make test passes — unit suite 36 passed / 21 skipped / 0 failed
  (BACKENDS="cpu_neon cpu_scalar metal"). Skips are the usual
  fixture- and hardware-gated ones.
• make MODE=asan test — same, 0 failed.
• make format-check — fails on src/archs/transformer/arch_ops.c,
  which this PR does not touch. The clang-format on PATH here is
  23.1.0 and CI uses llvm@22; under llvm@22 both arch_ops.c and this
  PR's only changed file are clean. Pre-existing version skew, not this
  change.

For perf/kernel changes, before/after numbers and host:

	before (=0)	after (=1)
prefill tok/s	see below	see below
decode tok/s	see below	see below

Superseded — see the results comment below. The campaign has since
finished and refutes the premise of this PR: in five of six models the
scatter inside one arm exceeds the difference between arms, and in the
sixth the lever is 0.5 % slower. The two 27B models disagree in sign.
The numbers that stood here came from a first campaign whose load gate was
broken and read 15–20 % low; they have been removed rather than left to be
quoted. Do not merge on throughput grounds.

Host: M1 Max, macOS 27.0, TARGET=mac-omp, Accelerate + libomp,
BACKENDS="cpu_neon cpu_scalar metal". BLASSetThreading resolves on this
system (checked with a standalone dlsym probe), so the lever does engage —
without that the A/B would measure nothing.

API impact

• No change to include/geist.h

🤖 Generated with Claude Code

[geisten]

Campaign finished — the numbers do not support this change

25 paired runs, 26 discarded for contention. pp256 + 32 decode steps,
interleaved A/B/A/B, load < 3 gate before each run. =1 is the lever on.

model	on (best)	off (best)	Δ	widest within-arm range	verdict
bonsai27b-pq2	12.97	13.72	−5.4 %	10.5 %	noise
qwen3.8-27b	12.52	11.51	+8.7 %	11.4 %	noise
qwen3.5-4b	73.08	75.39	−3.1 %	18.4 %	noise
gemma4-e2b	166.95	166.03	+0.6 %	11.0 %	noise
llama3.2-3b	111.48	112.29	−0.7 %	10.4 %	noise
smollm2-360m	759.72	763.64	−0.5 %	0.2 %	slower

In five of six models the scatter inside a single arm is larger than the
difference between arms. In the sixth — the only one where the runs are
tight enough to resolve half a percent — the lever is marginally slower.

The two 27B models, which are the reason the change exists, disagree in
sign: −5.4 % on Bonsai-PQ2_0 and +8.7 % on qwen3.8-Q4_K_M, each with a
within-arm range that covers its own delta. The +16 % decode figure on
qwen3.8 is not credible either — decode is one row and barely touches BLAS.

So the oversubscription the sample(1) profile shows is real, but removing
it does not buy measurable throughput at pp256. The profile counted threads
parked, and parked threads are cheap; the assumption that they were
costing wall-clock was mine, and the measurement does not bear it out.

Two corrections to this PR's description

1. The table in the body is contaminated and should be ignored. It came
   from a first campaign whose gate only checked load before each run.
   smollm2 prefill reads 622–682 t/s there and 759–764 t/s here — the
   earlier numbers were 15–20 % low across the board.
2. My fix to that gate was also wrong at first: checking load after
   the run measures the benchmark's own threads, since a 27B prefill drives
   the 1-min average to 7–27 by itself. Every run came back flagged and ten
   full 27B runs were discarded for nothing. The criterion now used is the
   harness's own best/worst spread across repeats, which is immune to this.

Recommendation

Do not merge on these numbers. The code is correct and does what it says —
BLASSetThreading resolves, the thread-local handling is right, and the
GEIST_BLAS_THREAD_MATCH=0 escape hatch works — but there is no measured
benefit to justify carrying it, and AGENT.md §6 asks for a benchmark gate
this does not pass.

I would rather close this and keep the negative result findable than leave a
perf PR open on a hypothesis the data refutes. Happy to do either.

The CPU prefill gap to the PrismML fork (13.6 vs 17.35 t/s at pp256, #429)
therefore remains unexplained, and thread oversubscription can be crossed
off the list of candidates.

🤖 Generated with Claude Code

[geisten]

Closing: the campaign above refutes the premise. No measurable throughput benefit at pp256, and in the one model tight enough to resolve half a percent the lever is slightly slower. The mechanism and the negative result stay findable here; thread oversubscription is crossed off the list of candidates for the CPU prefill gap in #429.

The branch perf/blas-thread-match is left in place — reopening is cheap if a future profile gives a reason to revisit.