Repository navigation
ggml-metal : fix unary kernels compile on older Metal toolchains - #31360
Merged
Merged
Conversation
Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
ggerganov
marked this pull request as ready for review
October 29, 2026 08:09
github-actions
Bot
added
ggml
This was referenced Oct 29, 2026
edwardyoon
pushed a commit
to edwardyoon/focus-llama
that referenced
this pull request
Oct 30, 2026
…l-org#31360) Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
This was referenced Oct 31, 2026
5 of 7 tasks
xkong-anaconda
added a commit
to AnacondaRecipes/llama.cpp-feedstock
that referenced
this pull request
Nov 2, 2026
llama.cpp 0.8.0 **Destination channel:** defaults ### Links - [PKG-19394](https://anaconda.atlassian.net/browse/PKG-19394) - [Upstream release](https://github.com/ggml-org/llama.cpp/releases/tag/v0.8.0) - [Upstream diff](ggml-org/llama.cpp@v0.7.0...v0.8.0) - [win-arm64 CUDA 13.4 off-CI build: logs and artifacts](https://drive.google.com/drive/folders/1z-GI6wHJ0VIly_vBpAGGm5sSmFPODNh0?usp=sharing) ### Explanation of changes: - Updated llama.cpp from 0.7.0 (b11669) to **0.8.0** (b11960). - **win-arm64 CUDA CI failure is expected:** win-arm64 CUDA 13.4 CI compiles, then fails creating its test environment with `nothing provides __cuda needed by libllama-0.8.0-cuda134_had9b3fb_100`. The win-arm64 CI workers have no GPU, so the `__cuda` virtual package needed by the CUDA tests is missing, and the libllama / llama.cpp / llama.cpp-tests CUDA test tasks fail for the same reason. This is an infrastructure limitation, not a recipe problem; the win-arm64 CPU variant builds and tests green in the same graph. As for previous releases, the CUDA variant was built and tested off-CI on a win-arm64 machine with an NVIDIA GPU (linked above): all 3 outputs build and pass their tests, ctest 35/35, `test-backend-ops -b CUDA0` 19192/19192, plus model inference, llama-server (including the web UI) and router mode. - ctest now selects tests labelled `main` or `behavior` and excludes `model` and `python`, matching whole words . Upstream relabelled most unit tests from `main` to `behavior`, so `-L main` alone would select only 8. The new selection keeps every previously run test and adds the two speculative-decoding tests: 42 → 44 on CPU, 39 → 40 on Metal, and 31 → 35 on Windows (which also gains test-generate-models and test-llama-context). On Metal, test-speculative-rejection-sampling is excluded: it hits the same `GGML_ASSERT(buf_dst)` on the CI Mac (macOS 13) as test-thread-safety and test-llama-context, and passes 3/3 with this PR's Metal package on an M4 Pro / macOS 15.7.3 once the shader fix below is applied. - Backported upstream [#31360](ggml-org/llama.cpp#31360) (`fix-metal-unary-bf16-constexpr.patch`). Without it the Metal package fails every context creation on macOS 14 and 15: the `constexpr bfloat4` casts new in 0.8.0 don't compile (upstream [#31272](ggml-org/llama.cpp#31272), [#31337](ggml-org/llama.cpp#31337)). Reproduced with this PR's CI Metal package on an M4 Pro / macOS 15.7.3. CI can't see it: the CI Mac runs macOS 13, where BF16 is off and this code never compiles. - llama-server's web UI is now built from source (`npm ci` on upstream's lockfile, then `vite build`; `nodejs` added to build) instead of downloading upstream's prebuilt bundle from a Hugging Face bucket (`LLAMA_USE_PREBUILT_UI`, default ON; 0.7.0 shipped the download too). The build fails if no UI was built, since upstream otherwise falls back silently to a UI-less server. `llama-ui-licenses.md` (Vite's license manifest, enabled by `ui-license-manifest.patch`) lists the licenses of the 179 bundled npm packages. - `test-backend-load` now requires a CUDA device on CUDA variants, so a broken CUDA backend can't pass on the CPU backend alone. - Smaller fixes: the Metal device fallback in `metal_gpu_selection.patch` no longer leaks the device array or releases devices it doesn't own (the file builds without ARC); `fix-models-path` and `disable-metal-flash-attention` are refreshed to apply without offsets; llama.cpp-tools also skips py>=3.15 (upstream `requires-python <3.15`); `openblas-devel` uses the `openblas_devel` CBC key. - New llama.cpp-tools test: fail if any `conversion`/`examples` import in the installed package was not redirected by `fix-convert_hf_to_gguf.patch`, including function-local ones such as `--dspark`. - gguf and llama.cpp-tools stay off win-arm64, with a corrected reason: the only win-arm64 transformers (5.20.2) constrains `mistral-common >=1.11.7`, which isn't on any channel yet (PKG-19294). gguf has no blocker of its own; it is skipped there only because it builds in the same output set. [PKG-19394]: https://anaconda.atlassian.net/browse/PKG-19394?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
xkong-anaconda
added a commit
to AnacondaRecipes/llama.cpp-feedstock
that referenced
this pull request
Nov 5, 2026
llama.cpp 0.9.0 **Destination channel:** defaults ### Links - [PKG-19636](https://anaconda.atlassian.net/browse/PKG-19636) - [Upstream release](https://github.com/ggml-org/llama.cpp/releases/tag/v0.9.0) - [Upstream diff](ggml-org/llama.cpp@v0.8.0...v0.9.0) ### Explanation of changes: - Updated llama.cpp from 0.8.0 (b11960) to **0.9.0** (b12110). - **win-arm64 CUDA CI failure is expected**: the GPU-less win-arm64 workers can't create the CUDA test environment. The CUDA 13.4 variant will be built and tested off-CI on a win-arm64 machine with an NVIDIA GPU, as for previous releases. - Dropped two patches that upstream now includes: `fix-metal-unary-bf16-constexpr.patch` (upstream [#31360](ggml-org/llama.cpp#31360)) and `link-test-backend-ops-threads.patch` (upstream [#31414](ggml-org/llama.cpp#31414) links `Threads::Threads` itself). - **gguf and llama.cpp-tools are built on win-arm64** - Release builds now set `-DLLAMA_BUILD_IS_DEV=OFF`. Upstream defaults to a dev build, so 0.8.0 shipped `llama.pc` and the CMake package versioned `0.8.0-dev` and `find_package(llama 0.8.0 EXACT)` failed. A new libllama test checks that the installed `llama.pc` reports exactly `{{ version }}`. [PKG-19636]: https://anaconda.atlassian.net/browse/PKG-19636?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
yhirose
added a commit
to yhirose/culebra
that referenced
this pull request
Nov 6, 2026
llama.cpp v0.8.0's unary.metal declares its bfloat casts constexpr, which the Metal compiler of macOS 14 and 15 rejects at run time, so on an Apple Silicon Mac running either, every Llama.load with the GPU failed with "failed to build 'unary' library" (ggml-org/llama.cpp#31360). The macOS CI runners are macOS 26, whose compiler accepts it. cpp-llamalib v0.9.0 pins its llama.cpp in one set(LLAMA_CPP_VERSION ...) line and reads it with scripts/llama-cpp-version.sh; check_llama_versions.sh now asks that script instead of reading GIT_TAG itself.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Remove
constexprfrom theggml_metal_unary_castbfloat specializations, which fail to compile with-Winvalid-constexpron the Metal toolchains of macOS 14 / 15, breaking Metal backend init (see #31272).Requirements
dsh:llama.cppMacBook, M5 Max, 128GB◷ 20.5m · ⚙ 3.6m · ↑ 76k · ↓ 56k · ⇈ 600.0 tok/s · ⇊ 51.3 tok/s · ≡ 212 · ⌨ 16