[bong-water-water-bong]

Engine #390, engine side only (no Flash-Next perf work). The llama.cpp half is already in the pin (#441 pins 8d9ce2c1a, which has fork #102): ggml-hrx's scheduler knows the backend of every node and can report the placement or refuse the graph. Nothing in 1bit serve asked it to, and nothing told the user.

What this adds

• --placement-report: at load, serve prints one line with the op types HRX placed on the CPU, their counts and the graph split count:

  1bit serve: HRX placement: 34 of 512 nodes off HRX (GET_ROWS x 4, MUL_MAT x 30), 7 graph splits
  

  and 1bit serve: HRX placement: all 512 nodes on HRX, 1 graph splits when nothing was left behind. The same numbers are the onebit_cpu_fallback_ops{op} series app/serve_metrics.h already derives from the backend's line. llama-server drops the library's INFO lines below -lv 4 (it maps GGML_LOG_LEVEL_INFO to its trace level, common/log.cpp), so the report never reaches serve at the default verbosity: this flag is what passes -lv 4 to the backend for that load. Without the flag nothing changes.

• --strict-device: refuse to start instead of running part of the graph on the CPU. serve sets GGML_HRX_STRICT=1, the backend aborts at load on the first node off HRX naming its op, type and shape, and serve exits non-zero with its own line instead of "did not become ready". If a report does arrive and names off-HRX nodes anyway, serve refuses with the same numbers. It needs --device hrx or auto; with --device cpu|ds4|onnx|npu the combination is refused, and an auto run that ends up with no HRX backend says so instead of passing silently.

Both flags only set what the backend already reads, and a GGML_HRX_REPORT / GGML_HRX_STRICT in the environment wins over what serve passes (0 turns either off), so an operator keeps control.

Unit test, CPU only (tests/hrx_placement_test.cpp, ctest hrx_placement, added to the CI build list): the observer's exact line in, then the node count, split count, off-HRX count and per-op counts out; the summary and the refusal text; unrelated lines ignored; a total-only "CPU fallback: N" line from another backend still counted; and ServeMetrics::placement() as the refusal's input. It also covers two parser defects the report exposes in the existing metrics reader: the first number after "off HRX" was taken as the total (for a clean graph that is the split count - 1 instead of 0), and the graph size and split count were not read at all.

Docs: docs/serve.md, "Off-HRX placement: --placement-report, --strict-device" (both lines, both flags, the verbosity reason, the env override, and the test).

Validation (CPU only, ryzen, nice -n 19, no GPU work)

cmake -B /tmp/t390-build -G Ninja -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_C_COMPILER=gcc-15 -DCMAKE_CXX_COMPILER=g++-15            # configure OK
cmake --build /tmp/t390-build --target onebit <every test target in ci.yml>   # rc=0, links 1bit
ctest --test-dir /tmp/t390-build --output-on-failure
    100% tests passed, 0 tests failed out of 32   (98.86 s)

hrx_placement is one of them; metrics_serve (the existing end-to-end check of
onebit_cpu_fallback_ops against tests/fake_backend.py, whose line has no "N off HRX" count) and
device_route still pass, so the parser change and the new flags change nothing for the routes
that were already there. python3 tools/copyright.py --check and actionlint on ci.yml are clean.

Not verified here: a real HRX run - the placement line, -lv 4 and the strict abort all need HRX0, and this box's GPU is under the orchestrator's queue. The strict path depends on the pinned backend honouring GGML_HRX_STRICT, which is what #441's pin carries; the engine side fails loudly if a graph is refused and is a no-op when the backend does not know the variable.

🤖 Generated with Claude Code

Hardware check (HRX0, strixhalo, 2026-10-18, 30 min slot): llama.cpp 1a5dcba2 (fork #102) and this patch, 1bit serve on Qwen3-0.6B-Q4_K_M and MiniCPM5-1B-Q4_K_M.

• --placement-report: the backend printed placement_observer: placement: 986 nodes, 0 off HRX (none), 1 graph splits (750/1 for the 1B) and serve printed 1bit serve: HRX placement: all 986 nodes on HRX, 1 graph splits, then came ready. The line only appears with the flag, which is the -lv 4 requirement holding in practice.
• No flags: ready, no placement line, no behaviour change.
• --strict-device: ready on both, which is correct for a clean graph, and serve notes that it saw no report (it does not force -lv 4).

Not exercised on hardware: the refusal itself. Both models on that box put every node on HRX, and the model #390 reports as falling back (Qwen3.8-Flash-Next UD-IQ4_XS) is not on strixhalo. The refusal is covered by the unit test (hrx_placement: off-HRX counts, the exact refusal line) and by fork #102's own GGML_ABORT path, not by a live off-HRX graph.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 390 - Partially compliant Compliant requirements: The --strict-device flag implements the strict mode, refusing to start if any node is off HRX The --placement-report flag prints a line at load with op types placed off HRX, their counts, and graph split count The implementation correctly parses and reports the backend's placement information Non-compliant requirements: The performance analysis and graph split count per token for Flash-Next is not covered in this PR Requires further human verification: The specific performance gap analysis for Flash-Next and graph split counts per token on HRX 441 - Partially compliant Compliant requirements: The PR includes the updated pins for hrx-system and llama.cpp The PR adds tests to validate the new functionality Non-compliant requirements: The PR does not include the full validation steps for the new pins Requires further human verification: Full validation of the new pins against the old ones 102 - Partially compliant Compliant requirements: The PR includes the necessary build changes for Windows The PR includes documentation for the ONNX backend on Windows Non-compliant requirements: The PR does not include the full testing of the ONNX backend on Windows Requires further human verification: Full testing of the ONNX backend on Windows

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Regex parsing logic The regex parsing logic for extracting placement information from backend logs may be fragile. Specifically, the logic that parses "34 off HRX" and "nodes" counts relies on specific string patterns and may not handle all edge cases or variations in backend log output. This could lead to incorrect parsing of placement data. static const std::regex off_count(R"((\d+) off HRX\b)"); const bool stated = std::regex_search(line, m, off_count); const long stated_off = stated ? std::stol(m[1]) : 0; if (!ops.empty()) { cpu_fallback_ops = ops; cpu_fallback_total = sum; } else if (!stated && std::regex_search(rest, m, number)) { cpu_fallback_total = std::stol(m[1]); // a total alone, with no per-op counts } if (stated) { cpu_fallback_total = stated_off; cpu_fallback_ops = ops; // empty for a clean graph ("none") ++placement_lines; } // the graph's size and its split count, when the line names them (engine #390) static const std::regex nodes_count(R"((\d+) nodes\b)"); static const std::regex splits_count(R"((\d+) graph splits\b)"); if (std::regex_search(line, m, nodes_count)) placement_nodes = std::stol(m[1]); if (std::regex_search(line, m, splits_count)) placement_splits = std::stol(m[1]); Strict device flag validation The validation for --strict-device flag only checks for specific device types (cpu, ds4, onnx, npu) but does not explicitly check for auto device. While the code implies that auto should be allowed, it's not explicitly validated, which could lead to unexpected behavior if auto is used with a non-HRX backend. if (o.strict_device && (o.device == "cpu" || o.device == "ds4" || o.device == "onnx" || o.device == "npu")) throw std::runtime_error("--strict-device needs --device hrx or auto: it refuses a graph that is not all on HRX");