…e it (#390) (#442)
The llama.cpp side is already in the pin (#441, llama.cpp fork #102): ggml-hrx's scheduler
knows the backend of every node, prints one line when GGML_HRX_REPORT is set, and aborts on
the first node off HRX with its name, op, type and shape when GGML_HRX_STRICT is. The engine
asked for neither, so a graph that partly ran on the CPU was invisible.
- --placement-report: passes -lv 4 to the backend for that load (llama-server drops the
library's INFO lines below verbosity 4, common/log.cpp) and prints serve's own line at
load: "1bit serve: HRX placement: 34 of 512 nodes off HRX (GET_ROWS x 4, MUL_MAT x 30), 7
graph splits". The same numbers are the onebit_cpu_fallback_ops series serve already read.
- --strict-device: sets GGML_HRX_STRICT=1 and refuses to start. A backend that refuses the
graph is reported as that, not as "did not become ready", and a report that still names
off-HRX nodes after a backend reached ready is a refusal with the same numbers. It needs
--device hrx or auto; an auto run with no HRX backend says so instead of passing silently.
- A GGML_HRX_REPORT / GGML_HRX_STRICT the user sets in the environment wins over serve's.
- app/serve_metrics.h: read the report's own count ("N off HRX": for a clean graph the first
number after the phrase was the split count, 1 instead of 0) and the graph's node and split
counts, which the metric series and the new line both use.
- tests/hrx_placement_test.cpp (ctest hrx_placement, listed in the CI build step): the
observer's line in, the counts, the summary and the refusal out; no GPU and no backend.
- docs/serve.md: "Off-HRX placement: --placement-report, --strict-device".
Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
scripts/build-windows.shnow also buildsryzenai-server.exe(third_party/ryzenai-server, the ONNX backend from #100) and places Microsoft's ONNX Runtime GenAI 0.11.2 and ONNX Runtime 1.23.2win-x64DLLs (MIT, pinned, sha256-checked) beside it.1bit.exethen serves ONNX Runtime GenAI model directories on Windows (CPU).How it builds with llvm-mingw. Upstream is untouched:
add_subdirectorys ryzenai-server and clears its MSVC-only/SUBSYSTEM:CONSOLElink flag;Wbemidl.hshim covers MinGW's lowercase header;onnxruntime-genai.dllthrough its import library.Not fetched: DirectML. DirectML (the GPU on Windows) needs
DirectML.dll, which is under Microsoft's redistributable licence rather than MIT, so the script leaves it out. docs/windows.md and docs/onnx.md say so.Tested under Wine 11 on Strix Halo, from a clean clone (the script ran in 2 min), with neither
--devicenor any backend path:1bit.exe servechoseonnx, startedryzenai-server.exebeside it and answered "The capital of France is Paris." On a GGUF it startedllama-server.exeon Vulkan and answered the same.taskkill /F /IM 1bit.exeleft nothing running.ryzenai-server.exealone: 61 tok/s on Qwen2.5-0.5B int4.🤖 Generated with Claude Code