hip: add community gfx1012 support with version-gated legacy compatibility

[CC-David-CC]

Updated to upstream 0.1.34 (1678de3). Fresh targeted builds and regressions passed; the full fleet/context performance numbers below remain measurements of the prior 0.1.33 source.

Rebase checks

This is the hardware-only split requested in #323, based on upstream
0.1.34 (1678de3). The tested card is the consumer AMD Radeon RX 5500 XT
8GB, gfx1012. It remains in the community/unvalidated architecture list.

Changes

• Admit gfx1012 for manual HIP builds.
• Signed-byte dot via RDNA1 SDWA, adapted from pinned llama.cpp/ggml with MIT
  attribution preserved.
• Version-gate legacy wave synchronization and host-allocation API names;
  HIP 7 retains its native operations and spellings.
• For hipBLAS 0.x, use the real rocBLAS handle/workspace API. Newer hipBLAS
  retains its existing implementation.
• Preserve Q2_0 FP16 signed zero on gfx1012/HIP below 7, with a regression test.

Shared serving and performance alternatives are separate contributions. The P4
tests worked using upstream's existing STRATA_EXPERIMENTAL_SM60; this branch
adds no SM61 flag, Pascal initializer or FP32 GEMM fallback.

Validation

Check	RX 5500 XT / HIP 5.7.1	RX 7900 XTX / HIP 7.15
Committed-source build	Pass	Pass
Intrinsics and wave/shared-memory synchronization	Pass	Pass
BF16/F16 prefill GEMM against CPU reference	Pass	Pass
1024-value signed-zero regression	Pass	Pass
Real-weight expert parity, layers 0/3/20/47	Pass	Pass
Upstream versus hardware-patch token/state identity	Upstream does not admit gfx1012	Pass, 8 requests per arm

The independent hardware build completed an 8K-input short code request at
15.30 generation tok/s, with 106.17 s prefill and 114.34 s total for 125
output tokens. Short prose was 11.68 tok/s, 118.14 s total for 218 output tokens.

It also completed 8192 input / 2591 output with 131072
total context tokens allocated, at 15.66 generation tok/s,
139.54 s prefill and 304.98 s total. This uses Q8 KV
streaming with 32768 resident cells. It is an allocation test, not a full
128K-input test.

The combined hardware/serving tree completed full 8K, 32K and 64K input in both
MTP modes. These combined measurements are labelled separately. Full 32K/64K
input was not repeated on the hardware-only tree, and full 128K input on the
5500 XT remains untested.

Hardware, scope and reproduction
| Measured results
| Machine-readable evidence

GSQ-RCO IQ3_S, Q8 KV, text only. Other RDNA1 cards, Windows HIP, vision and
multi-GPU behavior are untested. The older-host journal contained a workqueue
CPU-time warning; no GPU reset or memory-fault entry appeared in that window.

Credit: Niko1221/Strata supplies the engine,
expert cache and MTP. The RDNA1 dot implementation is adapted from pinned
llama.cpp/ggml code.

[Niko1221]

Merged for the next release: the gfx1012 admission, the version-gated legacy HIP names and __syncwarp, the hipBLAS 0.x path through rocBLAS, the SDWA dot (with its MIT attribution) and the Q2_0 signed-zero fix, with your commits and tests. Left out: the README banner, the fork's review and benchmark files, and the change that removed CMakeLists.txt's byte-order mark. gfx1012 is a by-hand build, listed in the new docs/OLDER_GPUS.md with your numbers. HIP 7 builds keep their native operations, and the gfx1201/gfx1030 Windows build is unchanged.

[earlvanze]

Additional gfx1012 / HIP 7 hardware confirmation: Qwen3.8-Flash-Next Coder IQ1_M successfully generated and ran a small Python example on an Intel iMac with 8 GiB AMD VRAM. Measured 2026-10-11. This is one short coding smoke test, not a context sweep or a comparison against the RX 5500 XT results above.

Hardware / software

• Apple iMac20,1; Intel Core i7-10700K (8 cores / 16 threads); 32 GB installed RAM (~31 GiB usable); internal Apple AP1024N NVMe SSD.
• AMD Navi14, PCI 1002:7340, Apple subsystem 0218; HIP reports gfx1012, wave32, 12 CUs, 7.984 GiB VRAM. Reporting the identifiers rather than guessing a retail GPU name.
• Omarchy 4.0.4; kernel 7.2.6-arch2-Watanare-T2-4-t2; amdgpu.
• AMD TheRock ROCm 7.14.0a20260608, official gfx101X-dgpu wheels; HIP driver/runtime report 71460850 / 71460850.
• Source build, engine 0.1.41, synced from checkout fa6c9af95d72a328b3527dbebdad93ca95b5e69b. HIP enabled, CUDA disabled, CMAKE_HIP_ARCHITECTURES=gfx1012, STRATA_PREFILL_MMQ=ON, STRATA_BUILD_TESTS=ON. Default SDWA signed-dot path; STRATA_GFX1012_PORTABLE_DOT left off.
• Device allocation self-test, hip_intrinsics, hip_prefill_gemm, and hip_prefill_native_batch all passed.

Model / settings

ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF, revision 5348543e0147355ac9cbcb031184a3546350988e, IQ1_M directory. Files: Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf and ...-00002-of-00002.gguf. This is the expert-pruned Coder release; the IQ1_M label should not be interpreted as every weight being 1-bit.

Launch command below uses portable path placeholders:

STRATA_RESIDENT_PIN=0 build/strata \
  --pack "$CODER_PACK" \
  --native "$CODER_SHARD_1" --ple-gguf "$CODER_SHARD_2" \
  --expert-profile data/expert-profile-coder.bin \
  --expert-cache auto --prefill auto \
  --spec 4 --spec-min-p 0.5 --mtp "$MTP_PACK" \
  --max-context 8192 --resident-experts --vram-reserve-mib 1024 \
  --mtp-draft-vocab data/draft_vocab_en.bin \
  --tokens-file prompt-tokens.txt --max-new 192 --stop-eos --temperature 0

Thinking disabled in the chat template. Default suffix drafting retained. Automatic prefill selected chunk size 256; expert pool used 7 workers plus the host thread. GPU cache held 797 experts. The pageable RAM expert cache held 20.08 GiB / 10,546 experts, with the remaining experts file-backed. Thus this is GPU + system RAM + file-backed offloading, not the entire model fitting in 8 GiB VRAM. Physical SSD traffic was not measured separately.

Workload / measured result

Fresh CLI process, no conversation-prefix reuse. Filesystem caches were not cleared; this was after setup and an exploratory run, so it is not a cold-disk benchmark. One measured run, 58 input tokens, output cap 192, stopped at EOS after 66 output tokens (including the end-of-turn token).

Prompt:

Write a Python function unique_sorted(values) that returns the unique integers in ascending order. Include assertions for [3, 1, 3, -2], [], and [5, 5]. Output only the Python code.

Engine timing lines:

decode                   66 tokens in 5229.8 ms  ->  12.62 tok/s
prefill                  57 tokens in 4424.4 ms  ->  12.88 tok/s  (time to first token 16300.8 ms)

Engine-reported TTFT: 16.30 s. Full subprocess wall time, including initialization/loading and generation: 26.56 s. The engine reports 57 prefill positions for the 58-token input; the counts above are preserved as logged.

The answer implemented return sorted(set(values)). All three requested assertions passed, plus an independent [9, 0, -10, 9] -> [-10, 0, 9] case: 4/4 checks passed.

Limits

Only this short prompt was measured; 8192 is the configured context capacity, not the tested input length. No repeated-run median, full 8K input, long-session stability, API serving, vision, multi-GPU, or broader coding-quality evaluation. PCIe link, power limits, background-load isolation, and peak process RAM/VRAM were not measured. No calibration or experimental speed projection was requested. An earlier exploratory run without --stop-eos continued into special tokens and is excluded from the throughput result.

[earlvanze]

Follow-up to my initial gfx1012 iMac Coder smoke test: memory-mapped experts plus a smaller VRAM reserve improved repeatability and time to first token on this 32 GB RAM / 8 GB VRAM desktop.

Same iMac20,1, i7-10700K, AMD Navi14 gfx1012 (7.984 GiB VRAM), internal NVMe, and HIP build at fa6c9af95d72a328b3527dbebdad93ca95b5e69b (engine 0.1.41). TheRock ROCm 7.14.0a20260608, HIP driver/runtime 71460850; PREFILL_MMQ enabled, default SDWA path. Full hardware/build details are in the original comment.

Model: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF, revision 5348543e0147355ac9cbcb031184a3546350988e, IQ1_M expert-pruned release. Same prepared pack, expert profile, MTP, and draft vocabulary as before.

Both changes were applied together:

--- baseline flags
+++ tuned flags
- --resident-experts --vram-reserve-mib 1024
+ --mmap-experts --vram-reserve-mib 512

This removes the separate resident expert RAM copy and leaves CPU experts backed by mmap / the OS page cache. GPU expert-cache capacity rose from 797 to 1057 experts. No code, driver, clocks, or sysctl changes.

Three interleaved baseline/tuned runs per workload, alternating arm order by repetition. Each run used a fresh CLI process. These are medians of the final comparison runs, after exploratory tuning:

Measurement	58-token input baseline	58-token input tuned	1,992-token input baseline	1,992-token input tuned
Decode (tok/s)	12.91	14.39	4.96	14.36
Prefill (tok/s)	12.62	6.94	82.00	106.15
Engine-reported TTFT (s)	16.806	8.505	36.796	19.123
Full subprocess wall time (s)	28.664	18.198	66.259	29.518

Short-prompt prefill throughput regressed even though TTFT and total wall time improved. The larger long-prompt gain coincided with severe baseline memory/swap pressure; this is not evidence of a general 2.9× speedup, and the two settings were not isolated from each other in this final comparison.

A local QMD llama.cpp embedding server was temporarily stopped for these paired measurements and restored afterward. Desktop/background activity remained, including file synchronization; filesystem caches were not cleared or controlled. Baseline automatic resident-cache allocations varied, with swap usage reaching roughly 19 GiB in a snapshot. This is a practical shared-desktop result, not an isolated cold-cache benchmark.

Every final comparison run

Workload	Arm	Repetition	Output tokens	Decode tok/s	Prefill tok/s	TTFT s	Wall s
58 input	baseline	1	66	12.91	12.56	14.543	24.210
58 input	candidate	1	69	13.35	6.86	8.623	18.901
1992 input	baseline	1	66	0.55	96.95	30.002	156.341
1992 input	candidate	1	66	11.51	107.83	18.966	29.518
58 input	candidate	2	69	16.02	9.82	6.062	15.390
58 input	baseline	2	66	13.52	13.02	16.990	28.664
1992 input	candidate	2	66	14.54	106.15	19.123	28.412
1992 input	baseline	2	66	6.12	82.00	36.796	55.096
58 input	baseline	3	66	5.30	12.62	16.805	34.645
58 input	candidate	3	69	14.39	6.94	8.505	18.198
1992 input	baseline	3	66	4.96	55.35	48.761	66.259
1992 input	candidate	3	66	14.36	100.81	20.044	29.617

Output counts include EOS. The short tuned answer adds a # Assertions comment (69 versus 66 output tokens); outputs are not byte-identical. Engine prefill counts were 57 / 1,991 tokens. Wall time includes process initialization and loading.

Reproduction settings: --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --max-context 8192 --max-new 192 --stop-eos --temperature 0, default 7 expert-pool workers plus host thread, and STRATA_RESIDENT_PIN=0. Thinking disabled in the chat template. Keep the original model/pack/profile/MTP/draft-vocab arguments, replacing only the two flags above.

Short request:

Write a Python function unique_sorted(values) that returns the unique integers in ascending order. Include assertions for [3, 1, 3, -2], [], and [5, 5]. Output only the Python code.

The longer input adds synthetic reference comments before the same task. It is not a realistic large-codebase workload. Construction with the same tokenizer/template (request is the text above):

filler = "Reference data for an unrelated module; ignore it for the requested function.\n" + "".join(
    f"# record {i}: input values are integers; preserve the public interface and deterministic ordering.\n"
    for i in range(150)
)
filler = tok.decode(tok.encode(filler)[:1930])
content = filler + "\n\nTASK:\n" + request
prompt = template.render([{"role": "user", "content": content}], enable_thinking=False)
tokens = tok.encode(prompt, parse_special=True)

All six tuned comparison outputs reached EOS and passed four functional cases: the three requested assertions plus [9, 0, -10, 9] -> [-10, 0, 9]. Baseline outputs also passed. This verifies a small function, not broad coding quality; tuning and confirmation used the same task.

After restoring the embedding server, one additional short smoke test using the saved tuned launcher achieved 14.31 tok/s, 6.801 s engine TTFT, and 17.243 s full wall time, with all four checks passing. The embedding endpoint also successfully returned a real embedding. This extra run is separate from the medians above.

An observed tuned graph-capture snapshot had 334 MiB VRAM free; this is not a peak-memory measurement. Context capacity was configured to 8,192, but full 8K inputs and sustained/concurrent serving have not been validated. The 512 MiB reserve is a measured option for this machine/workload, not a proposed universal default.