Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ body:
id: device
attributes:
label: Device
options: [auto, npu, hrx, cpu, ds4, onnx, not sure]
options: [auto, npu, hrx, cpu, ds4, not sure]
validations:
required: true
- type: input
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/model_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ body:
attributes:
label: Device you want it on
multiple: true
options: [npu, hrx, cpu, ds4, onnx, any]
options: [npu, hrx, cpu, ds4, any]
validations:
required: true
- type: textarea
Expand Down
6 changes: 0 additions & 6 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,6 @@
url = https://github.com/1bit-MONSTER/lemonade.git
branch = main
shallow = true
[submodule "third_party/ryzenai-server"]
path = third_party/ryzenai-server
url = https://github.com/lemonade-sdk/ryzenai-server
# DwarfStar (docs/dwarfstar.md): our fork 1bit-MONSTER/ds4, branch 1bit/main = upstream
# antirez/ds4 main plus our commits (1BP packages, docs/1BP.md there); built by
# scripts/build-ds4.sh; .github/workflows/bump-ds4.yml merges upstream into 1bit/main.
[submodule "third_party/ds4"]
path = third_party/ds4
url = https://github.com/1bit-MONSTER/ds4.git
Expand Down
2 changes: 1 addition & 1 deletion .pr_agent.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ persistent_comment = true
extra_instructions = """
This is the 1bit engine: a C++26 inference engine behind an OpenAI-compatible API (`1bit serve`), running inside
Lemonade, on AMD Strix Halo's XDNA 2 NPU (fast lane, the from-scratch dx kernels), the Radeon iGPU (HRX
through a pinned llama.cpp build with HIP kernels; no Vulkan, ROCm backend, Metal, ZINC or CUDA), DwarfStar and ONNX. Review like an engineer who runs this on the hardware:
through a pinned llama.cpp build with HIP kernels; no Vulkan, ROCm backend, Metal, ZINC or CUDA) and DwarfStar. Review like an engineer who runs this on the hardware:
- correctness first: tensor shapes and strides, KV-cache positions and rollback, bf16/f32 rounding (dx kernels
round to nearest even), NPU hardware-context and runlist lifetimes, lifetimes of child processes and ports;
- C++: ownership and RAII, no allocations in per-token loops, exceptions reaching main, thread safety of shared state;
Expand Down
21 changes: 1 addition & 20 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ if(ONEBIT_HRX)
foreach(_dev hrx cpu)
add_test(NAME serve_e2e_${_dev}
COMMAND ${CMAKE_SOURCE_DIR}/tests/serve_e2e.sh $<TARGET_FILE:onebit> ${ONEBIT_SERVE_TEST_GGUF} ${_dev})
# hrx skips without a gfx1151 (ONEBIT_HRX_E2E_TARGETS), as onnx_npu_smoke does without an NPU
# hrx skips without a gfx1151 (ONEBIT_HRX_E2E_TARGETS)
set_tests_properties(serve_e2e_${_dev} PROPERTIES SKIP_RETURN_CODE 77)
endforeach()
endif()
Expand All @@ -227,25 +227,6 @@ if(ONEBIT_DS4)
target_compile_definitions(onebit PRIVATE ONEBIT_DS4_SERVER="${_onebit_ds4_server}")
endif()

# ── ONNX Runtime GenAI (docs/onnx.md) ──────────────────────────────────────
# Off by default: builds ryzenai-server (MIT) with scripts/build-onnx.sh as its own program;
# `1bit serve --device onnx` runs it as a child process.
option(ONEBIT_ONNX "Build ryzenai-server (third_party/ryzenai-server) for --device onnx" OFF)
if(ONEBIT_ONNX)
set(ONEBIT_ONNX_SERVER "${CMAKE_BINARY_DIR}/onnx/ryzenai-server")
add_custom_target(onnx_server
COMMAND ${CMAKE_SOURCE_DIR}/scripts/build-onnx.sh ${CMAKE_BINARY_DIR}/onnx
BYPRODUCTS ${ONEBIT_ONNX_SERVER}
USES_TERMINAL)
add_dependencies(onebit onnx_server)
onebit_build_relative(ONEBIT_ONNX_SERVER _onebit_onnx_server)
target_compile_definitions(onebit PRIVATE ONEBIT_ONNX_SERVER="${_onebit_onnx_server}")
# Linux NPU through Ryzen AI Software (docs/onnx.md): skips without ONEBIT_RYZENAI_ROOT,
# an NPU and a model (ONEBIT_ONNX_NPU_MODEL)
add_test(NAME onnx_npu_smoke COMMAND ${CMAKE_SOURCE_DIR}/tests/onnx_npu_smoke.sh $<TARGET_FILE:onebit>)
set_tests_properties(onnx_npu_smoke PROPERTIES SKIP_RETURN_CODE 77)
endif()

# ── Hugging Face tokenizers (docs/tokenizers.md) ───────────────────────────
# Off by default: it needs cargo. Any model's tokenizer.json, through our C ABI
# over third_party/tokenizers (the same release as the tokenizers npm/PyPI
Expand Down
23 changes: 0 additions & 23 deletions NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -119,29 +119,6 @@ Lemonade (third_party/lemonade)
built as its own program (scripts/build-lemonade.sh) that runs `1bit serve`
as a backend. None of its code is linked into this product.

ryzenai-server (third_party/ryzenai-server)
https://github.com/lemonade-sdk/ryzenai-server
Copyright (c) 2026 lemonade-sdk. License: MIT.
Built as a separate program (scripts/build-onnx.sh) that `1bit serve --device
onnx` runs as a child process; none of its code is linked into this product.
Its build fetches cpp-httplib (MIT), nlohmann/json (MIT) and the ONNX Runtime
GenAI headers (MIT). The release package carries the built program in onnx/,
with its licence as onnx/ryzenai-server-LICENSE.txt. The AMD_LICENSE file in
its repository covers AMD's Ryzen AI Software, which this product does not ship
or copy: the NPU and hybrid modes use the user's own install (RYZENAI_INSTALL_PATH
at build time, ONEBIT_RYZENAI_ROOT at run time; docs/onnx.md, "Linux NPU"), and
the release build never builds against it (scripts/weekly.sh checks the package).

ONNX Runtime GenAI and ONNX Runtime
https://github.com/microsoft/onnxruntime-genai
https://github.com/microsoft/onnxruntime
Copyright (c) Microsoft Corporation. License: MIT.
scripts/build-onnx.sh fetches the release libraries (ONNX Runtime GenAI 0.11.2,
ONNX Runtime 1.23.2, pinned by sha256) and places them beside ryzenai-server.
The release package carries libonnxruntime-genai.so and libonnxruntime.so in
onnx/, with each project's LICENSE and ThirdPartyNotices.txt beside them
(onnx/onnxruntime-genai-*.txt, onnx/onnxruntime-*.txt).

Hugging Face tokenizers (third_party/tokenizers)
https://github.com/huggingface/tokenizers
Copyright Hugging Face, Inc. and the tokenizers authors
Expand Down
3 changes: 0 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,6 @@ hands to 1bit, it runs the engine as one of its backends, behind an OpenAI-compa
same build on the CPU.
- **The NPU engine** on the XDNA 2 NPU, in progress: correctness first, speed next
([docs/npu.md](docs/npu.md)).
- **ONNX Runtime GenAI** (Lemonade's ONNX format) on the CPU, and on the NPU through the user's
own install of AMD's Ryzen AI Software, on Linux
([#386](https://github.com/1bit-MONSTER/engine/pull/386), [docs/onnx.md](docs/onnx.md)).
- **DwarfStar**, for DeepSeek V4 Flash, GLM 5.x and Qwen3.8-Flash-Next in its own GGUFs, with SSD
expert streaming ([docs/dwarfstar.md](docs/dwarfstar.md)).
- **Laya**, opt-in, which picks where each request runs ([docs/laya.md](docs/laya.md)), and
Expand Down
2 changes: 1 addition & 1 deletion app/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
// 1bit: the engine binary. Subcommands:
//
// 1bit serve -m <model> one model on any device (NPU, HRX, CPU,
// DwarfStar, ONNX) behind an OpenAI-compatible API;
// DwarfStar) behind an OpenAI-compatible API;
// how the engine runs inside Lemonade (docs/serve.md).
// 1bit unified -m <model dir> one native model on the NPU fast lane behind
// OpenAI endpoints; serve's NPU route.
Expand Down
Loading
Loading