Move HRX pins: hrx-system to AMD's e3e6a976 line, llama.cpp 8d9ce2c1a by bong-water-water-bong · Pull Request #441 · 1bit-MONSTER/engine

bong-water-water-bong · 2026-10-18T02:23:37Z

The owner decided to move off the held HRX line (2026-10-18).

Pins

third_party/hrx-system 3319e536 → d3c731e1 (1bit-MONSTER/hrx-system 1bit/e3e6a976-pin-6: ae206d5e on 1bit/e3e6a976-pin, plus our hrx-system Steps 1-2: embedded Lemonade, and HRX + Vulkan in one llama.cpp build #6 replayed on top). This is AMD's e3e6a9766ccb plus our commits replayed on top: HRX_AQL_BLOCK_SIZE / HRX_COMMAND_BUFFER_MODE, the HRX: hipModuleLoadData segfaults on a truncated code object where ROCm returns an error #409, HRX: libhip_intercept wraps only 331 of 540 HIP symbols (rest silently unobservable) #410 and HRX opens the NPU device (/dev/accel/accel0) in GPU-only processes #401 fixes, and our **[Loom]: Multi-unit live-out demand dropped across loop back-edges miscompiles long-context flash-attention on gfx1151** ROCm/hrx-system#1535 live-out fix (AMD's guard is still in their source). The pin is a -s ours merge of 3319e536, so every shipped commit stays reachable and check_pins sees it as ahead. Its tree is byte-identical to the tested commit fc2213a1c.
third_party/llama.cpp 1a5dcba2 → 8d9ce2c1a (1bit-MONSTER/llama.cpp 1bit/hrx-bump-e3e6a976). This merges AMD's hrx-graph-develop-v2 43b0ec2c4 into our line, keeping our ggml-hrx. It also renames the corpus predicate mul( to multiple_of( and adds a bound on the ARGSORT rank.

Already measured on strixhalo 2026-10-17 against the old pins: test-backend-ops -b HRX0 1141/1141 x3; long-prompt needles 20/20, 5/5, 5/5; FA stress identical; KLD 0 and identical output on Qwen3-0.6B and ZAYA1-8B; pp512 +4.4% / +7.8%, tg128 +2.5% / +2.8%.

Final validation, on strixhalo at engine main d344381 with exactly these pins, against the old pins (performance mode):

Builds: onebit, llama_hrx, test-backend-ops, llama-server, llama-bench, llama-perplexity, libamdhip64 and libhip_intercept all build.
Steps 1-2: embedded Lemonade, and HRX + Vulkan in one llama.cpp build #6 stubs: generate_stubs.py --check against the built libamdhip64.so: "up to date: 540 reference symbols, 214 generated forwarders, node set matches amdhip64.map", exit 0.
Op suite: test-backend-ops -b HRX0 1141/1141 x2; hw-gate ops 611/611.
MUL_MAT flake: -o MUL_MAT 347/347 x10 on both pins. The single case f32 m=1 n=1 k=2048 failed 1/201 on the new pins and 2/201 on the old. It is the same kernel on both (common.mul_mat.f32_f32_decode), so this is the known HRX: intermittent MUL_MAT f32 m=1 n=1 k=2048 error on common.mul_mat.f32_f32_decode (about 1 in 40 runs at fork tip e44c9d0) #370 flake, not a regression.
Long prompts: needle 7341 @4732 20/20, @4432 5/5, Qwen3-1.7B 5/5; tests/hrx_long_prompt.py 5/5; FA stress 0/20 bad. No NaN lines anywhere.
Hardware gate: scripts/hw-gate.sh passes ops, needle, and kat on qwen3-0.6b, zaya1-8b and qwen3.8-27b. The four 0.6B kat warnings are identical on the old pins. Its KLD step reports FAIL only because the box has no recorded KLD base files, so it fails for any pin.
KLD by hand (old pins as the base, the gate's arguments): mean KLD 0.000000, 100% same top token and identical perplexity on all three models (22.5232 / 41.8533 / 6.6848).
Serving: 1bit serve --device hrx gives 5/5 identical temp-0 outputs on Qwen3-0.6B and on ZAYA1-8B; tests/serve_e2e.sh hrx and cpu both PASS.
Not covered: hrx-backend-test doesn't build on either pin. That is HRX: hrx-backend-test cannot build at fork tip ad18320a (deleted dispatch-symmetric-i4.h still included) #422, being fixed on this line.

1bit/hrx-vulkan-patched has been fast-forwarded to 8d9ce2c1a, and tag patched-1a5dcba2 keeps the old tip.

The hold stays, now naming d3c731e1, so later AMD hrx-system moves stay manual. hrx-system fork #6 (generate_stubs.py canonical signatures) is replayed on top as a clean cherry-pick. It was never in a shipped pin. The validation also runs generate_stubs.py --check against the built libamdhip64.so.

🤖 Generated with Claude Code

@4732

… (owner, 2026-10-18) third_party/hrx-system 3319e536 -> ae206d5e (1bit-MONSTER/hrx-system 1bit/e3e6a976-pin): AMD's e3e6a9766ccb with our commits replayed on it (HRX_AQL_BLOCK_SIZE / HRX_COMMAND_BUFFER_MODE, the engine#409, #410 and #401 fixes) and our fix for ROCm/hrx-system#1535. AMD's newer Loom no longer reaches #1535 on our prompts, but its multi-unit live-out guard is still in unit_liveness_builder.c, so we keep removing it. The pin is a `-s ours` merge of 3319e536 (every shipped commit stays reachable, check_pins sees it ahead) whose tree is the tested commit fc2213a1c. third_party/llama.cpp 1a5dcba2 -> 8d9ce2c1a (1bit-MONSTER/llama.cpp 1bit/hrx-bump-e3e6a976): AMD-Ecosystem hrx-graph-develop-v2 43b0ec2c4 merged into our line (118 conflicts, all under ggml-hrx, ours kept), the kernel corpus predicate renamed `mul(` -> `multiple_of(` (195 predicates, 54 files; e3e6 parses only multiple_of), and a range bound on the ARGSORT rank our small_rows_f32.loom stores through (the new Loom store-origin check rejected it, 46 ARGSORT cases). Measured on strixhalo 2026-10-17 against the old pins (performance mode, ~/wt/hrx-bump-test/REPORT.md): test-backend-ops -b HRX0 1141/1141 x3; needle 7341 Qwen3-0.6B @4732 20/20, @4432 5/5, Qwen3-1.7B @4732 5/5; FA stress 20x48 identical; KLD 0.000000 and identical 80-token output on Qwen3-0.6B and ZAYA1-8B; pp512 +4.4% / +7.8%, tg128 +2.5% / +2.8% (0.6B / ZAYA1-8B, medians of 3 interleaved rounds). config/hrx-system-hold.config now names the new pin (moves stay manual) and config/hrx-long-prompt-validated-pin.config records it. registry/architectures.json: only the llama.cpp pin field changes (292 architectures before and after). docs/hrx.md and the bump-hrx.yml comment describe the new line. Not in this pin: hrx-system fork PR #6 (c9656af, generate_stubs.py canonical signatures; only ever on 1bit/main, never in a shipped pin); it regenerates stubs_generated.c, so it gets its own replay and checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-18T02:24:25Z

PR Reviewer Guide 🔍

(Review updated until commit `0b64a38`)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 6 - Partially compliant Compliant requirements: Vendored Lemonade v11.9.0, complete, plus local deltas in its UPSTREAM.md Runs Lemonade's server in-process as `1bit lemonade` Includes smoke test (CI): health, catalog and system info Non-compliant requirements: None Requires further human verification: None 409 - Partially compliant Compliant requirements: Fix `hipModuleLoadData` segfaults on truncated code objects where ROCm returns an error Ensure bounds checking is not disabled when `data_length == 0` Non-compliant requirements: None Requires further human verification: None 410 - Partially compliant Compliant requirements: Ensure `libhip_intercept` wraps all 540 HIP symbols exported by `libamdhip64.so` Make all HIP symbols observable through the proxy Non-compliant requirements: None Requires further human verification: None
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review Outdated Documentation The documentation in `docs/hrx.md` still references the old `98d05d94` commit for the hrx-system hold, even though the PR moves the engine to AMD's `e3e6a9766ccb` with the fixes applied. This can cause confusion for developers and users who rely on the documentation to understand the current state of the HRX pins. \|---\|---\|---\| \| `third_party/llama.cpp` \| [1bit-MONSTER/llama.cpp](https://github.com/1bit-MONSTER/llama.cpp), branch `1bit/hrx-vulkan-patched` \| AMD's `hrx-graph-develop-v2` (ggml-hrx on ggml-org llama.cpp) plus our commits ("Our patches") \| \| `third_party/hrx-system` \| [1bit-MONSTER/hrx-system](https://github.com/1bit-MONSTER/hrx-system), pinned on branch `1bit/e3e6a976-pin-6` (the commit `config/hrx-system-hold.config` names; `1bit/main` is where the unheld bump merges AMD) \| [ROCm/hrx-system](https://github.com/ROCm/hrx-system) (libhrx, loomc and the Loom tools) at AMD's pin plus our commits ("Our hrx-system commits") \| Documentation Mismatch The `config/hrx-system-hold.config` file still references the old commit `3319e536196b99119f30f78b60f39a340aee6a5d` instead of the new commit `d3c731e1d7f47de2d64739b1ec298f0d0fe829d4`. This inconsistency can lead to confusion and incorrect assumptions about the current state of the HRX system. # commit rather than being emptied. # # 2026-10-16, R1 backport (owner decision): the hold now names `1bit/held-98d05d94-3fixes`,

Replays hrx-system fork #6 (c9656af, generate_stubs.py takes HIP entry point signatures from api.h so --check is trustworthy; regenerated stubs_generated.c) on top of ae206d5e as a clean cherry-pick: 1bit-MONSTER/hrx-system 1bit/e3e6a976-pin-6 = d3c731e1. #6 was only ever on the fork's 1bit/main, never in a shipped pin. The hold and the validated-pin record name d3c731e1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-18T02:25:35Z

Persistent review updated to latest commit e8133e9

github-actions · 2026-10-18T03:32:22Z

Persistent review updated to latest commit e8133e9

…in-6 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-18T03:33:57Z

Persistent review updated to latest commit 0b64a38

bong-water-water-bong · 2026-10-18T04:36:30Z

Two checks on this pin, both from the bump test's open items (strixhalo ~/wt/hrx-bump-test/REPORT.md). Nothing here changed a pin; no PR beyond this comment.

What was run

line	engine	hrx-system	llama.cpp	`llama-server --version`
new (this pin)	`fce3a687c1dd101d39562b268c4ad94efc45e4ca`	`d3c731e1d7f47de2d64739b1ec298f0d0fe829d4`	`8d9ce2c1aa60ed6b2334df3576e203e40c533a9c`	`10643 (8d9ce2c1a)`
old (comparison)	`6dd709f7774bfb74c845d8d5c84b5f93c6852bea`	`3319e536196b99119f30f78b60f39a340aee6a5d`	`1a5dcba2aa6d91dacec3c3c00a37dd77aa9c6084`	`1 (1a5dcba)`

Built with -DONEBIT_HRX=ON from ~/wt/t433-current (new) and the existing ~/wt/engine-main-game tree (old), under ~/lb/thermal-run.sh. Boot amdgpu.cwsr_enable=0. Both box locks held for every GPU step, one job at a time, Tctl pause above 88 C, and a per-line GGML_HRX_JIT_CACHE_DIR so one line's compiled kernels cannot leak into another's.

1. MUL_MAT f32 k=2048

bin/test-backend-ops -b HRX0 -o MUL_MAT        # 347 MUL_MAT cases per run, HRX0 only

pin	warm JIT cache	cold JIT cache (deleted before each run)	MUL_MAT failures
new	10 runs, 347/347 each	10 runs, 347/347 each	0
old	5 runs, 347/347 each	5 runs, 347/347 each	0

The case the bump test caught is in that set and passed on every run, on both pins:

MUL_MAT(type_a=f32,type_b=f32,m=1,n=1,k=2048,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1): OK

The single failure in the bump test was in a full-suite run, which a MUL_MAT-only run cannot cover, so I also ran the whole suite on the new pin:

pin	full suite `test-backend-ops -b HRX0`	failures
new	3 runs, 1141/1141 each	0

So: not reproduced in 30 focused runs (20 new, 10 old, warm and cold JIT cache alike) nor in 3 full-suite runs on the new pin. The 1-in-7 event in the bump test was a MUL_MAT f32 m=1 n=1 k=2048 case reporting ERR 1.58e-3 > 5e-4; if it recurs, the two things to capture are that case line verbatim and the GGML_HRX_JIT_CACHE_DIR it ran with — a cold-cache, first-execution fault is the one shape this run cannot rule out, because it is also the one shape that did not happen in 30 attempts.

2. Long-context needle beyond 4.7K (the #1535 shape)

New pin's llama-server on Qwen3-0.6B-Q4_K_M.gguf (qwen3.context_length = 40960 in the GGUF, so 16K is inside the trained context), -dev HRX0 -ngl 99 -c 20480 -np 1 --jinja, requests with cache_prompt:false, ~/wt/loom-cog/needle_n.py PORT 5 <lines> (vault code 7341):

prompt	length	reps	pass	answer	sha256/8
540 lines	8032 tokens	5	5/5	`7341`	`e3768df7`
1100 lines	16532 tokens	5	5/5	`7341`	`e3768df7`

Server log: 0 NaN lines. So the defect is masked on this pin beyond 4.7K as well, not only at the bump test's 4732.

Box state after the runs

To re-run

export IREE_HAL_AMDGPU_LIBHSA_PATH=/opt/rocm-therock/lib/python3.14/site-packages/_rocm_sdk_core/lib/libhsa-runtime64.so.1
export GGML_HRX_JIT_CACHE_DIR=<per-line dir>

~/wt/t433-current/build/hrx/llama/bin/test-backend-ops -b HRX0 -o MUL_MAT    # 347 cases, ~8 s
~/wt/t433-current/build/hrx/llama/bin/test-backend-ops -b HRX0              # 1141 cases

~/wt/t433-current/build/hrx/llama/bin/llama-server -m ~/models/Qwen3-0.6B-Q4_K_M.gguf \
  -dev HRX0 -ngl 99 -c 20480 -np 1 --jinja --host 127.0.0.1 --port 18771 --no-webui &
python3 ~/wt/loom-cog/needle_n.py 18771 5 540     # 8032 tokens
python3 ~/wt/loom-cog/needle_n.py 18771 5 1100    # 16532 tokens

Logs and per-run summaries: strixhalo:~/wt/hrx-zaya-nan/tpin/out/ (run.sh, full.sh alongside).

🤖 Generated with Claude Code

…e it (#390) (#442) The llama.cpp side is already in the pin (#441, llama.cpp fork #102): ggml-hrx's scheduler knows the backend of every node, prints one line when GGML_HRX_REPORT is set, and aborts on the first node off HRX with its name, op, type and shape when GGML_HRX_STRICT is. The engine asked for neither, so a graph that partly ran on the CPU was invisible. - --placement-report: passes -lv 4 to the backend for that load (llama-server drops the library's INFO lines below verbosity 4, common/log.cpp) and prints serve's own line at load: "1bit serve: HRX placement: 34 of 512 nodes off HRX (GET_ROWS x 4, MUL_MAT x 30), 7 graph splits". The same numbers are the onebit_cpu_fallback_ops series serve already read. - --strict-device: sets GGML_HRX_STRICT=1 and refuses to start. A backend that refuses the graph is reported as that, not as "did not become ready", and a report that still names off-HRX nodes after a backend reached ready is a refusal with the same numbers. It needs --device hrx or auto; an auto run with no HRX backend says so instead of passing silently. - A GGML_HRX_REPORT / GGML_HRX_STRICT the user sets in the environment wins over serve's. - app/serve_metrics.h: read the report's own count ("N off HRX": for a clean graph the first number after the phrase was the split count, 1 instead of 0) and the graph's node and split counts, which the metric series and the new line both use. - tests/hrx_placement_test.cpp (ctest hrx_placement, listed in the CI build step): the observer's line in, the counts, the summary and the refusal out; no GPU and no backend. - docs/serve.md: "Off-HRX placement: --placement-report, --strict-device". Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

…awn_worker, census, #433) (#446) * docs: the 2026-10-17/18 merges (HRX pins, DwarfStar, PM router and spawn_worker, census, #433) The docs still described the pre-2026-10-18 line. - README status: HRX pinned to AMD e3e6a976 + llama.cpp 8d9ce2c1a with the measured gate numbers; DwarfStar pinned to 5b013dc1bb75 (1BP v5) and its hardware check; the PM gains --pm-router laya and --pm-spawn-url; the census numbers of 2026-10-17. - docs/PORTING.md: the HRX pin (step 2), the Laya source/revision bump (step 4), the 2026-10-17 census (step 5). - docs/registry.md: the 2026-10-17 census table, against registry/census.json, and the current unmapped list. - docs/hrx.md: the engine#433 finding next to the other HRX0 known issues (no abort in 60 fresh loads; the 458-token prompt cannot come from this model template). - docs/dwarfstar.md: what the 2026-10-17 pin was checked on, and that serve cannot set ds4 expert cache size. - .github/workflows/bump-hrx.yml: the loom-jit trap is now described as the step that guards it, with #441 as the validated hand move and #422 still open. Facts from merged PRs #441, #429, #438, #439, #434, #436 and the hardware comments on #441 (TPIN) and #429 (TDS4). * docs/hrx.md: the LoomC-symbol guard, which is the loom-jit trap, and #441 as the validated hand move The workflow comment that described the trap cannot be pushed with this token (no workflow scope), so the fact lives where the pin process is documented: docs/hrx.md, next to the hold. --------- Co-authored-by: pi-zayap2 <bcloud@localhost>

github-actions Bot added the Review effort 3/5 label Oct 18, 2026

This was referenced Oct 18, 2026

HRX: crafted ELF header with small e_shoff still crashes inside the AMDGPU HAL executable loader #420

Closed

HRX: hrx-backend-test cannot build at fork tip ad18320a (deleted dispatch-symmetric-i4.h still included) #422

Closed

bong-water-water-bong marked this pull request as ready for review October 18, 2026 03:31

docs/hrx.md, hold config: the pin sits on fork branch 1bit/e3e6a976-p…

0b64a38

…in-6 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

bong-water-water-bong merged commit fce3a68 into main Oct 18, 2026
10 checks passed

bong-water-water-bong deleted the 1bit/hrx-pin-e3e6a976 branch October 18, 2026 03:37

This was referenced Oct 18, 2026

serve: report off-HRX placement at load, and --strict-device to refuse it (#390) #442

Merged

HRX: report or refuse CPU fallback (strict mode); Flash-Next UD-IQ4_XS 18x slower than a peer engine #390

Closed

bong-water-water-bong mentioned this pull request Oct 18, 2026

docs: the 2026-10-17/18 merges (HRX pins, DwarfStar, PM router and spawn_worker, census, #433) #446

Merged

This was referenced Oct 18, 2026

HRX: hrx-backend-test and test-hrx-ops abort on stale kernel-routing expectations #444

Closed

census: new HF model class uncovered by the registry #447

Closed

Navigation Menu

Repository navigation

Move HRX pins: hrx-system to AMD's e3e6a976 line, llama.cpp 8d9ce2c1a - #441

Move HRX pins: hrx-system to AMD's e3e6a976 line, llama.cpp 8d9ce2c1a#441
bong-water-water-bong merged 3 commits into
mainfrom
1bit/hrx-pin-e3e6a976

bong-water-water-bong commented Oct 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

Uh oh!

bong-water-water-bong commented Oct 18, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

bong-water-water-bong commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Reviewer Guide 🔍

(Review updated until commit 0b64a38)

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

Uh oh!

bong-water-water-bong commented Oct 18, 2026

What was run

1. MUL_MAT f32 k=2048

2. Long-context needle beyond 4.7K (the #1535 shape)

Box state after the runs

To re-run

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

bong-water-water-bong commented Oct 18, 2026 •

edited

Loading

github-actions Bot commented Oct 18, 2026 •

edited

Loading

(Review updated until commit `0b64a38`)