Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/PORTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Each step below is one PR (or a short series) that builds and runs on Strix Halo
| 2 | **HRX.** llama.cpp with `GGML_HRX=ON` on AMD's tested pair (it was built with `GGML_VULKAN=ON` too until RFC #213 stage 3, which removed the engine's Vulkan and ROCm builds) | `1bit-MONSTER/llama.cpp` `1bit/hrx-vulkan-patched` (AMD's `hrx-graph-develop-v2` plus our commits) + `1bit-MONSTER/hrx-system` `1bit/main` (`ROCm/hrx-system` at AMD's pin plus our commits), pinned from `ROCm/ggml-staging-automation` | **landed** ([docs/hrx.md](hrx.md)): `1bit serve` runs GGUF checkpoints on `HRX0` (and the CPU from the same build); kept current by `bump-hrx.yml`. The decode-split flash-attention race (#123, #140) is fixed (llama.cpp `00adc2b`, #176) and the kernel is on by default again (#178). Since 2026-10-18 pinned to AMD's `e3e6a976` hrx-system line (`1bit/e3e6a976-pin-6`: our four fixes replayed plus our live-out fix for ROCm/hrx-system#1535) with llama.cpp `8d9ce2c1a` (#441; since 2026-10-19 hrx-system `82804197` and llama.cpp `00f2a68de`, #458, #461). Measured at the #441 move: test-backend-ops 1141/1141, long-prompt needles 20/20 + 5/5 + 5/5, KLD 0 and identical output on Qwen3-0.6B and ZAYA1-8B, pp512 +4.4% / +7.8%, tg128 +2.5% / +2.8%. The hold in `config/hrx-system-hold.config` now names that pin, so later AMD hrx-system moves stay manual |
| 3 | **NPU engine.** The NPU engine, with its kernels built from source | 1bit-MONSTER's NPU engine | **in progress** ([docs/npu.md](npu.md)): the model reader, Q4NX, the tokenizer and serving landed (#8, #9), and Qwen3-0.6B was served on the NPU through `1bit serve`. The XDNA driver is pinned upstream (#12). GGUF on the NPU (#179, #181, #209): `1bit serve -m <gguf> --device npu` repacks a GGUF to Q4NX and runs six architectures, correct but not yet fast. The layer kernel and lm head built from source decode Qwen3-0.6B, 1.7B and 4B at cosine similarity 0.9999 or better against an fp64 reference at every tested position, and greedy decode matches the reference exactly; they are closed source and not merged yet |
| 3x | **35B MoE on the NPU (experimental, closed source).** Qwen3.6-35B-A3B on the NPU | a closed-source add-on | **landed as an add-on** ([docs/npu.md](npu.md#add-on-routes)): built into `1bit` with `-DONEBIT_NPU_PRIVATE`; parity against the fp64 reference passes (3 positions, argmax 846 / 198 / 3710), and `1bit serve --device npu` answers "The capital of France is Paris." Without the add-on, `serve` says the route is not part of the build |
| + | **Linux kernel.** The kernel that provides `amdxdna` and `amdgpu`, pinned to upstream | `torvalds/linux` release tags; config from the Strix Halo kernel of 2026-09-23 | **moved** to its own repository (2026-10-04): v7.3-rc4 built into Debian packages with `amdxdna` in-tree, and that repository keeps it current. Installing it on Strix Halo is a separate, deliberate step; 1bit OS (`os/`) takes a built kernel (`KVER=` / `VMLINUZ=`) |
| + | **Linux kernel.** The kernel that provides `amdxdna` and `amdgpu`, pinned to upstream | `torvalds/linux` release tags; config from the Strix Halo kernel of 2026-09-23 | **moved** to its own repository (2026-10-04): upstream tags built into Debian packages — `v7.4-rc1` today, `v7.3-rc4` before it, with `amdxdna` in-tree — and that repository keeps it current. Installing it on Strix Halo is a separate, deliberate step; 1bit OS (`os/`) takes a built kernel (`KVER=` / `VMLINUZ=`) |
| 4 | **Laya router.** A non-autoregressive scorer that picks where each request runs | `src/laya_scorer.cpp`, `include/laya_scorer.h` on `backup/laya-and-results-2026-09-22`; model at `~/models/laya` | **landed, opt-in** ([docs/laya.md](laya.md)): source `NandhaKishorM/laya` + the three HF checkpoints pinned, hash-verified fetch, `bump-laya.yml` (#18; source `ca5045a3f1fa` and checkpoint revision `7b928d828b7b` since #439); the C++ scorer matches the Python reference on the root and `typed-decisions/` checkpoints (max logit diff 8.6e-6, same argmax) and `1bit serve --device auto --laya-model <dir>` routes each request (#90). Load 2.8 s once, then 0.38 s per decision (was 8.75 s, #91). Since RFC #186 (#187) it classifies each conversation (95.5% on 200 labelled requests) and `config/route-policy.json` picks the device; the scorer runs on HRX at 15-16 ms a decision (#206, #208). Next: `multilingual/` (mmBERT-base) |
| 5 | **Every HF model, kept current.** The architecture registry and the daily HF census that finds new architectures and ranks the unmapped ones by model count (1bit-MONSTER's registry mapped 2,030 HF arch strings to its own kernels; here the backends' pinned code decides) | `src/model_registry*.cpp`, `Testing/census_*.py` and `.json`, `.github/workflows/census-{watch,sweep,autopr}.yml` | **landed** ([docs/registry.md](registry.md)): `registry/architectures.json` generated from the pinned HRX llama.cpp fork (265 HF architectures at first; ZINC's column left with the core strip); `census.yml` sweeps HF daily. First sweep: 415,414 text-generation models, mapped 94.88%, checked 64.33% of those with an architecture (reported separately); the 2026-10-17 sweep counts 425,483 text-generation models, 338,650 with an architecture (2,822 architectures), 94.62% mapped and 63.9% checked (#438). All seven checked GGUF architectures pass on HRX since the llama.cpp pin `96f6b89` (#95) |
| 5z | **ZAYA1 (Zyphra) on the GPU.** An architecture upstream llama.cpp lacks, in our llama.cpp; `1bit serve --device hrx` (and `auto`) runs it | `1bit-MONSTER/llama.cpp` `1bit/hrx2-archive` (`src/models/zaya.cpp`) and 1bit-MONSTER's `tools/convert_zaya_safetensors_to_gguf.py` | **landed** ([vulkan.md at 0baf286](https://github.com/1bit-MONSTER/engine/blob/0baf286/docs/vulkan.md#zaya1-zyphra-from-our-llamacpp)): ported to the pinned fork with a converter; matches transformers in FP32 at 95/96 teacher-forced positions; Q4_K_M decoded at 93 tok/s on the removed Vulkan build, 61.5 on the removed ROCm build and 25.5 on HRX at first; on HRX it now decodes at about 90 tok/s (94.0 tg128, llama.cpp #42, [docs/hrx.md](hrx.md#not-yet)) and passes `serve_e2e` |
Expand Down
6 changes: 3 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,9 +70,9 @@ exits cleanly, so Lemonade can restart the model

## The kernel

The box's Linux kernel is ours too: `v7.3-rc4` plus the IOMMU PerfOpt backports, built from our
config, MOK-signed for Secure Boot, and gated like a pin bump. Moving it to the next upstream tag is
a written step list ([kernel.md](kernel.md)).
The box's Linux kernel is ours too: `v7.4-rc1`, plain upstream since the IOMMU PerfOpt series is
native there, built from our config, MOK-signed for Secure Boot, and gated like a pin bump. Moving it
to the next upstream tag is a written step list ([kernel.md](kernel.md)).

## Pinned sources

Expand Down
26 changes: 26 additions & 0 deletions docs/kernel.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,32 @@ list, with the facts that go stale marked as facts and the commands that do not.
"The XDNA driver") and `ec_su_axb35` (the box's fan EC). Both live in
`/lib/modules/<version>/updates/` and both must be rebuilt against the new kernel and signed.

## 7.4 on the box: what changed, and what to watch

`v7.4-rc1` is the first tag where the **PerfOpt series is upstream** (`iommu/amd` plus the `amdgpu`
identity-domain enable), so nothing is carried: the tree is the plain tag, and the boot still reports
`AMD-Vi: PerfOpt armed on IOMMU0`. Built from the same config with `CONFIG_DEBUG_INFO_BTF=y`
(pahole 1.31, ~5 MiB of BTF in the image), which is also why the `bpf-restrict-fs: Failed to load BPF
object` boot noise is gone — systemd now logs `LSM BPF program attached`. Compiler is **gcc 15.2.0**,
the same as the 7.3 image; older notes that say clang were wrong.

- **Throughput, 7.4 arm only:** Qwen3.8-27B UD-Q4_K_XL on HRX0, five repeats, at the current
90/90/105 W fan curve — pp512 239.6, pp2048 234.7, tg128 10.9 tok/s (medians). These do **not** line up
with the 7.3 table above, which was taken on a different machine and power profile; the 7.3 arm of
*this* recipe cannot be taken from inside a 7.4 boot (it needs its own one-shot `grub-reboot` window),
so no 7.3-vs-7.4 delta is claimed. Greedy output is byte-identical run to run, and the NPU
known-answer probe is bit-identical against its fp64 reference.
- **Watched, not caused by 7.4:** the root NVMe logs `AMD-Vi: Event logged [IO_PAGE_FAULT]` under block
I/O (read-only translations, no `nvme` error / timeout / reset, no filesystem error, `smartctl` clean) —
a device-side read-after-unmap class of noise, monitored rather than normalised.
- **Watched, and specifically not a 7.4 regression:** a MES `REMOVE_QUEUE` timeout that ends in a MODE2
GPU reset. The same chain is recorded on the 7.3 kernel four days earlier, and the MES wait budget is
unchanged between the two tags; on this box it has appeared with system memory exhausted and several
resident GPU clients, which is the regime to keep the box out of.
- The two out-of-tree modules were rebuilt against 7.4 and signed the same way, and the NPU path loads
our `amdxdna` build, not the kernel's own copy — the installed `updates/` module wins the resolution
(`modinfo -n amdxdna`), which is the check that proves it after a reboot.

## What "ready" means

1. an upstream tag to move to, and whether the PerfOpt series is **native** in it (then the
Expand Down
8 changes: 5 additions & 3 deletions docs/os-v3.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ The screen covers the *whole* install, which for v3 means one list that spans bo
1 target disk and identity /dev/nvme0n1 YMTC PC41Q-2TB 1.9 TB user onebit host 1bit-strixhalo
2 base system (Ubuntu 26.04 Server) curtin: partitions, unpack, packages, bootloader
3 our repository from the stick 8 debs: kernel, modules, rocm, xrt, engine, lemonade, appliance, keyring
4 kernel and modules vmlinuz-7.3.0-rc4-1bit-perfopt+, amdxdna, ec_su_axb35, initramfs
4 kernel and modules vmlinuz-7.4.0-rc1-1bit, amdxdna, ec_su_axb35, initramfs
5 Secure Boot key already enrolled | queued for MokManager at the next boot
6 firmware boot entry 1bit OS -> \EFI\ubuntu\shimx64.efi BootOrder 0000,...
7 data partition adopted 1BIT-DATA (1.1 GB, 4 models) | created empty
Expand Down Expand Up @@ -317,7 +317,7 @@ Ubuntu entry, failed, and fell through to the stick.
Three fixes, all in the installed system rather than in a special-case installer:

1. **efivarfs exists.** Our kernel is built with `CONFIG_EFIVAR_FS=m` (measured in
`/boot/config-7.3.0-rc4-1bit-perfopt+`) and an Ubuntu base loads it, so `/sys/firmware/efi/efivars` is
`/boot/config-7.4.0-rc1-1bit`) and an Ubuntu base loads it, so `/sys/firmware/efi/efivars` is
populated (97 entries on this box). The v2 image never staged the module and BusyBox never loaded it,
which is why the live installer could not fix anything from *inside* the installed system either
(engine#709). v3 installs from a full Ubuntu session, where `efibootmgr` works, and asks for `=y`
Expand Down Expand Up @@ -470,7 +470,9 @@ a console; everything after it is headless.

## What is measured now

Reference box (strixhalo, 2026-11-01, the machine this design was written on):
Reference box (strixhalo, 2026-11-01, the machine this design was written on). These are the readings
the design was written against and stay as the record; the box and the stick have run
`7.4.0-rc1-1bit` since 2026-11-04 (plain `v7.4-rc1`, `CONFIG_EFIVAR_FS=m` there too).

| fact | measured |
|---|---|
Expand Down
2 changes: 1 addition & 1 deletion docs/releases.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ DeepSeek V4.x, GLM 5.x and Qwen3.8-Flash-Next run on **HRX** here, and the catal
| hrx-system | `82804197bcbe` |
| HRX toolchain (ROCm/TheRock SDK) | `10.3.0a20261031`, `gfx1151` (`config/hrx-toolchain.config`), HSA runtime `21a6682452e159cf…` |
| Lemonade Server | `2026.46.0~1.ef0f3b49` (our fork, the `onebit` recipe) |
| kernel in the image | `7.3.0-rc4-1bit-perfopt` (IOMMU PerfOpt backports, MOK-signed) |
| kernel in the image | `7.4.0-rc1-1bit` (Linux `v7.4-rc1`, IOMMU PerfOpt upstream, MOK-signed) |
| NPU kernel sets | 3 sealed sets, 30 kernels, every manifest signed |

The pinned toolchain's HSA runtime, so a bump is a diff here and a wrong runtime is visible
Expand Down
4 changes: 2 additions & 2 deletions os/v3/puppy/kernel-puppy.fragment
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
#
# Measured 2026-11-01 (docs/os-v3-live-puppy-eval.md): Puppy's initrd mounts the medium and layers the
# save file with these drivers and filesystems itself, so they must be in the image, not in a module the
# initrd has no way to load yet. Our 7.3.0-rc4-1bit-perfopt+ has them as modules (Ubuntu's shape) and
# initrd has no way to load yet. Our `7.4.0-rc1-1bit` has them as modules (Ubuntu's shape) and
# the initrd stops 2.1 s into the boot with
#
# *** /puppy_upup64n_24.04.sfs not found. ... dropping out to console
Expand All @@ -31,7 +31,7 @@
# --- the medium: CD (the ISO in QEMU, and a stick written as an ISO) -------------------------------
# the chain UNDER the next four lines, and the reason they are here: kconfig will not let a tristate be
# `=y` while a symbol it depends on is `=m` - `make olddefconfig` does not fail, it quietly reduces the
# child back to `=m`. Ours are all `=m` today (measured in /boot/config-7.3.0-rc4-1bit-perfopt+: SCSI,
# child back to `=m`. Ours are all `=m` today (measured in /boot/config-7.4.0-rc1-1bit: SCSI,
# SCSI_MOD, SCSI_COMMON, CDROM, ATA, SATA_AHCI), so a fragment that set only the drivers would be
# silently half-applied - the trap the per-symbol check in os/v3/puppy/README.md exists to catch.
CONFIG_SCSI=y # [m] every one of the medium drivers below depends on this
Expand Down
5 changes: 3 additions & 2 deletions os/v3/puppy/mk-huge-kernel.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,9 @@
#
# The 1bit OS kernel as a Puppy kernel package: woof-CE's `kernel-kit/output/huge-<kver>.tar.*`,
# which `3builddistro` unpacks into `vmlinuz` and the `zdrv_<prefix>_<version>.sfs` it boots. This is
# how the Puppy build carries OUR kernel (7.3.0-rc4-1bit-perfopt+ and its out-of-tree modules), with no
# distribution kernel anywhere in the image - docs/os-v3-live-puppy-eval.md, and the owner's requirement
# how the Puppy build carries OUR kernel (`7.4.0-rc1-1bit` today, built from upstream `v7.4-rc1`, and
# its out-of-tree modules), with no distribution kernel anywhere in the image - docs/os-v3-live-puppy-eval.md,
# and the owner's requirement
# that the Puppy appliance boots our kernel, our modules and our firmware.
#
# <kernel-stage> is the release kernel as it sits on the build machine, the same shape os/v2/mkdebs.sh
Expand Down
Loading