Skip to content

Windows: ryzenai-server.exe (--device onnx) in build-windows.sh - #102

Merged
bong-water-water-bong merged 2 commits into
mainfrom
win/onnx
Sep 25, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
win/onnx

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

scripts/build-windows.sh now also builds ryzenai-server.exe (third_party/ryzenai-server, the ONNX backend from #100) and places Microsoft's ONNX Runtime GenAI 0.11.2 and ONNX Runtime 1.23.2 win-x64 DLLs (MIT, pinned, sha256-checked) beside it. 1bit.exe then serves ONNX Runtime GenAI model directories on Windows (CPU).

How it builds with llvm-mingw. Upstream is untouched:

  • a wrapper project add_subdirectorys ryzenai-server and clears its MSVC-only /SUBSYSTEM:CONSOLE link flag;
  • a one-line Wbemidl.h shim covers MinGW's lowercase header;
  • the MinGW-built exe links Microsoft's MSVC-built onnxruntime-genai.dll through its import library.

Not fetched: DirectML. DirectML (the GPU on Windows) needs DirectML.dll, which is under Microsoft's redistributable licence rather than MIT, so the script leaves it out. docs/windows.md and docs/onnx.md say so.

Tested under Wine 11 on Strix Halo, from a clean clone (the script ran in 2 min), with neither --device nor any backend path:

  • On a CPU ONNX model directory, 1bit.exe serve chose onnx, started ryzenai-server.exe beside it and answered "The capital of France is Paris." On a GGUF it started llama-server.exe on Vulkan and answered the same.
  • taskkill /F /IM 1bit.exe left nothing running.
  • ryzenai-server.exe alone: 61 tok/s on Qwen2.5-0.5B int4.

🤖 Generated with Claude Code

bong-water-water-bong and others added 2 commits September 25, 2026 18:48
… ONNX Runtime GenAI and ONNX Runtime

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Sep 25, 2026

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit d074203

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

100 - Partially compliant

Compliant requirements:

  • Cross-builds the engine for Windows 10+ x64 on Linux
  • Builds ryzenai-server.exe with ONNX Runtime GenAI and ONNX Runtime DLLs
  • Includes Microsoft's ONNX Runtime GenAI 0.11.2 and ONNX Runtime 1.23.2 win-x64 DLLs
  • Documents the build process and usage in docs/windows.md and docs/onnx.md
  • Does not fetch DirectML as it is under Microsoft's redistributable licence
  • Tests under Wine 11 on Strix Halo

Non-compliant requirements:

  • The PR does not explicitly state that 1bit.exe serves ONNX Runtime GenAI model directories on Windows (CPU) in the PR description or code diff

Requires further human verification:

  • Whether 1bit.exe correctly serves ONNX Runtime GenAI model directories on Windows (CPU) after the build
  • Whether the binary correctly carries DT_RPATH $ORIGIN so ONNX Runtime GenAI's run-time dlopen finds the copy beside it
⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Missing Error Handling for Unzip Operations

The unzip_to function in scripts/build-windows.sh does not check if the unzip command succeeded. If the unzip operation fails, the script will continue without error, potentially leading to missing or corrupted files needed for the build. This could cause build failures or runtime issues.

unzip_to() { [ -d "$2" ] || { mkdir -p "$2.tmp" && (cd "$2.tmp" && cmake -E tar xf "$1") && mv "$2.tmp" "$2"; }; }
unzip_to "$(fetch $OGA_WIN $OGA_WIN_SHA oga-win.zip)" "$work/oga-win"
unzip_to "$(fetch $ORT_WIN $ORT_WIN_SHA ort-win.zip)" "$work/ort-win"
Potential Race Condition in CMake Build

The script builds ryzenai-server.exe using a wrapper project that modifies the CMake configuration. If multiple builds are running concurrently, there's a potential race condition in the creation of temporary directories ($work/rz-wrap, $work/rz-shim, $work/oga-root) or in the CMake build process itself. This could lead to inconsistent builds or failures.

mkdir -p "$work/rz-wrap" "$work/rz-shim" "$work/oga-root"
echo '#include <wbemidl.h>' > "$work/rz-shim/Wbemidl.h"
cat > "$work/rz-wrap/CMakeLists.txt" <<'RZ'
cmake_minimum_required(VERSION 3.20)
project(ryzenai-server-mingw CXX)
add_subdirectory(${RYZENAI_SERVER_SRC} rz)
set_target_properties(ryzenai-server PROPERTIES LINK_FLAGS "")
RZ
cmake -S "$work/rz-wrap" -B "$work/rz" -DRYZENAI_SERVER_SRC="$root/third_party/ryzenai-server" \
    -DCMAKE_TOOLCHAIN_FILE="$work/toolchain.cmake" -DCMAKE_BUILD_TYPE=Release -DOGA_ROOT="$work/oga-root" \
    -DOGA_INCLUDE="${oga}include" -DOGA_LIB="${oga}lib/onnxruntime-genai.lib" -DCMAKE_EXE_LINKER_FLAGS=-static \
    "-DCMAKE_CXX_FLAGS=-D_WIN32_WINNT=0x0A00 -I$work/rz-shim" > "$work/rz.cmake.log"
cmake --build "$work/rz" -j"$jobs" > "$work/rz.build.log" 2>&1 || { tail -20 "$work/rz.build.log"; exit 1; }

@bong-water-water-bong
bong-water-water-bong merged commit dffb58e into main Sep 25, 2026
5 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the win/onnx branch September 25, 2026 21:54
bong-water-water-bong added a commit that referenced this pull request Oct 16, 2026
… report) (#424)

One commit on top of the pin: llama.cpp PR #102, already verified on gfx1151.
The pair with the held-pin backport (hrx-system 3319e5361) is being built to
confirm before merge.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
bong-water-water-bong added a commit that referenced this pull request Oct 18, 2026
…e it (#390) (#442)

The llama.cpp side is already in the pin (#441, llama.cpp fork #102): ggml-hrx's scheduler
knows the backend of every node, prints one line when GGML_HRX_REPORT is set, and aborts on
the first node off HRX with its name, op, type and shape when GGML_HRX_STRICT is. The engine
asked for neither, so a graph that partly ran on the CPU was invisible.

- --placement-report: passes -lv 4 to the backend for that load (llama-server drops the
  library's INFO lines below verbosity 4, common/log.cpp) and prints serve's own line at
  load: "1bit serve: HRX placement: 34 of 512 nodes off HRX (GET_ROWS x 4, MUL_MAT x 30), 7
  graph splits". The same numbers are the onebit_cpu_fallback_ops series serve already read.
- --strict-device: sets GGML_HRX_STRICT=1 and refuses to start. A backend that refuses the
  graph is reported as that, not as "did not become ready", and a report that still names
  off-HRX nodes after a backend reached ready is a refusal with the same numbers. It needs
  --device hrx or auto; an auto run with no HRX backend says so instead of passing silently.
- A GGML_HRX_REPORT / GGML_HRX_STRICT the user sets in the environment wins over serve's.
- app/serve_metrics.h: read the report's own count ("N off HRX": for a clean graph the first
  number after the phrase was the split count, 1 instead of 0) and the graph's node and split
  counts, which the metric series and the new line both use.
- tests/hrx_placement_test.cpp (ctest hrx_placement, listed in the CI build step): the
  observer's line in, the counts, the summary and the refusal out; no GPU and no backend.
- docs/serve.md: "Off-HRX placement: --placement-report, --strict-device".

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant