[bong-water-water-bong]

Follow-up to #206. The router's GGUF scorer now runs on HRX0 (HRX + LOOM), not Vulkan. The engine no longer builds a Vulkan scorer.

• third_party/ggmlc now points at our fork, 1bit-MONSTER/ggmlc 1bit/main (df08d1b0). It adds an HRX build against a llama.cpp ggml, and gives every graph a uid. Without the uid, HRX re-imported and re-recorded the whole graph on every call, because ggml only assigns uids inside ggml_backend_sched.
• third_party/llama.cpp is bumped to 22ceda3a (fork Documentation site on GitHub Pages (1bit-monster.github.io/engine) #31). That brings the LOOM kernels ModernBERT needs on HRX: LayerNorm, strided/broadcast arithmetic, clamp, F32->F16 copy, per-head attention, odd-K matmuls, plus fused RoPE and GEGLU. test-backend-ops passes on HRX0. ZAYA1-8B on HRX has the same PPL with the new kernels on or off.
• cmake/laya-ggml.cmake (-DONEBIT_LAYA_GGML=ON) builds laya with TheRock's amdclang against third_party/llama.cpp's ggml and hrx-system.
• serve --laya runs laya daemon <gguf> --family typed-decisions --device hrx, with the HSA runtime HRX needs. tests/laya_gguf_route.sh checks both, now 9 checks.

Verified on Strix Halo from a clean clone of this branch (-DONEBIT_LAYA_GGML=ON), with 1bit serve --laya and no overrides: 95.5% on tests/laya_route_cases.json, p50 28 ms per request.

Scorer	Accuracy	Decision
C++ scorer, CPU	95.5%	538 ms
laya on HRX0	95.5%	15-16 ms at a steady 64 tokens, 25 ms at 128
laya on Vulkan (upstream ggmlc, not built)	95.5%	12 ms / 24-31 ms

docs/laya.md also records that the multilingual checkpoint routes worse than typed-decisions in all six languages measured (69.1% vs 83.7% over 682 translated cases), so the engine doesn't use it.

🤖 Generated with Claude Code

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit e98463c

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit e98463c)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 206 - Partially compliant Compliant requirements: Build ggmlc's laya with HRX backend instead of Vulkan Use the typed-decisions GGUF model for routing decisions Pin the ggmlc fork at 1bit/main with HRX build support Update cmake/laya-ggml.cmake to build laya with HRX support Update documentation to reflect HRX backend usage Verify performance improvements on Strix Halo Non-compliant requirements: None Requires further human verification: None 31 - Partially compliant Compliant requirements: Build documentation site at 1bit-monster.github.io/engine Use repository's README.md and docs/*.md files Implement clean, simple layout with sidebar and dark mode Ensure no broken internal links Build with tools/site.py and deploy via GitHub Actions Non-compliant requirements: None Requires further human verification: None

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Incorrect device parameter handling The LayaDaemon class now accepts an environment vector, but the device parameter is still passed as a command-line argument. This could lead to inconsistencies if the device parameter is also set via environment variables, as the command-line argument will take precedence. The device parameter should be handled consistently either through environment variables or command-line arguments, not both. explicit LayaDaemon(const std::vector<std::string>& argv, const std::vector<std::string>& env = {}) { int in[2], out[2]; if (::pipe2(in, O_CLOEXEC) != 0 || ::pipe2(out, O_CLOEXEC) != 0) throw std::runtime_error(std::string("laya: pipe: ") + std::strerror(errno)); std::signal(SIGPIPE, SIG_IGN); // a daemon that died fails a write, not the server std::vector<char*> args; for (const auto& a : argv) args.push_back(const_cast<char*>(a.c_str())); args.push_back(nullptr); const pid_t parent = ::getpid(); pid_ = ::fork(); if (pid_ < 0) throw std::runtime_error(std::string("laya: fork: ") + std::strerror(errno)); if (pid_ == 0) { ::prctl(PR_SET_PDEATHSIG, SIGTERM); if (::getppid() != parent) ::_exit(127); for (const auto& e : env) { const size_t eq = e.find('='); if (eq != std::string::npos) ::setenv(e.substr(0, eq).c_str(), e.c_str() + eq + 1, 0); } ::dup2(in[0], 0); ::dup2(out[1], 1); ::execvp(args[0], args.data()); ::_exit(127); } Missing error handling for HSA library path The code sets the IREE_HAL_AMDGPU_LIBHSA_PATH environment variable but does not check if the provided path is valid or accessible. If the HSA library path is incorrect, the laya daemon may fail to start or run with incorrect settings without clear error messages. if (const std::string hsa = hrx_libhsa(o.hrx_libhsa); !hsa.empty()) env.push_back("IREE_HAL_AMDGPU_LIBHSA_PATH=" + hsa); Hardcoded toolchain path The ONEBIT_HRX_TOOLCHAIN is hardcoded to /opt/rocm-therock. This makes the build process less portable and could break on systems where ROCm is installed elsewhere. It should be configurable via a CMake variable or environment variable. set(ONEBIT_HRX_TOOLCHAIN "/opt/rocm-therock" CACHE PATH "ROCm/TheRock root: its amdclang builds llama.cpp and HRX")

[github-actions]

Persistent review updated to latest commit e98463c