Repository navigation
Conversation
The four gfx1100 community issues (Niko1221#1655 Niko1221#1459 Niko1221#1406 Niko1221#1292) were read in full and triaged to this box; the useful part is in scratch/reports/COMMUNITY-KNOBS.md. Measured here at 16K, matched budgets: base 1,870.9 tok/s +STRATA_RAM_ADAPT=1 1,950.7 (+4.3%) +STRATA_PA_FAST=1 +PF_SWITCH_MIN_T=4096 1,979.0 (+5.8%) +STRATA_PF_GEMM=1 +PF_PAD=1 1,990.1 (+6.4%) +--prefill 32768 1,919.0 (+2.6%) +STRATA_PF_FUSED=1 1,898.2 (+1.5%) Combined (the three winners) 2,012.9 at 16K (+6.4%) and 945.3 at 4K (-0.1%); a 32K chunk adds nothing on top. All these are bit-changing switches, so switches-accuracy.py grades answers off vs on: the 16K needle passes in both and the one differing case ('strata' backwards) is flaky off AND on (3/5 each) - the 2-bit model, not the switches. Everything adopted; 73K real text through the server is now 1,784.3 tok/s (+7.8% today, +41% over the session's 1,264.5). Two corrections to the docs, from measurement: the WMMA/SIMT switches labelled gfx12 act on gfx1100, and on-device kernel-shape autotuning was a no-op here exactly as Niko1221#1292 reports. Also fixed, after this box hard-froze: stop-server.bat killed a fixed list of old binary names, so newer A/B engines leaked (~12 GiB each) until free RAM hit 0.0 GiB. It now kills any strata*.exe plus tune_decode, and the lessons (one engine at a time, budget as a validity gate, the RAM tier being the top lever) are in the strata-local-tuning skill.
664793c to
18f620b
Compare
|
Re-measured on the current line — the branch is rebased onto v0.1.43.1 and both rows now carry a fresh median-of-3 sweep on the shipped config (2026-10-13, same methodology: one-shot, greedy, 256 generated tokens, 3 clean cells per tier, stalls recorded and excluded — none today):
The short tier is the noisiest (1K prefill spans 652–820 — it is dominated by start-up state, ~25% spread); 32K–128K prefill sits in a ±1–2% band and decode in a ±3% band or tighter. The medians table above is what the docs rows carry; the ranges are in the DETAILS footnotes. vs the 0.1.38-era row this PR carried: decode +23% to +37% per tier (59.2 → 81.3 @128k), prefill 32K-128K +29% to +42%. The decode row includes |
18f620b to
7c5ae50
Compare
… — median of 3 clean cells per tier, the shipped config Engine 0.1.43.1 (refresh of the 0.1.38-era row), ROCm 10.2 nightly, the tuned gfx1100 hipBLASLt-100500 table (Niko1221#755), Ryzen 9 7900X / 96 GB, one-shot greedy runs, 256 generated tokens, median of 3 clean cells per tier (the recorded methodology). Decode now carries STRATA_ROUTE_TAIL_SKIP=7 (adopted after a task-battery-gated A/B; answers differ slightly, =0 restores) — the decode row moves 59.2-65.1 -> 71.4-81.3 tok/s; prefill 32K-128K moves 1,494-1,641 -> 2,117-2,149 tok/s.
7c5ae50 to
055fe9b
Compare
|
Follow-up: the 262K column is now filled too — measured on the same shipped config (0.1.43.1 + the hipBLASLt table + TAIL_SKIP=7 + lookup-chain 2): prefill 2,043 tok/s [2,038–2,051], decode 69.3 tok/s [67.9–71.4], median of 3 one-shot cells at a 261,888-token prompt (trimmed to fit the native 262,144 cap) + 256 generated. That tier had been "-" in every AMD row so far. |
|
Thanks for the pull request. We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected. To reopen it takes about two minutes:
The rules are in CONTRIBUTING.md. |
2 participants
Re-file of #319 — that PR was auto-closed on 2026-10-06 when this repository's main history was cleaned up (per the maintainer's note there, not a rejection). Same three commits, re-based onto the new main.
One correction the re-file made: the original branch's first commit carried a stale copy of
docs/DETAILS.mdalongside the intended row, so its diff showed ~190 lines of unrelated deletions of upstream text. This refile carries exactly the rates-row payload — the whole branch diff is +12/−2, the two removed lines being the caption lead lines the AMD captions extend. Original discussion: #319.Adds one IQ3_S (AMD RX 7900 XTX, gfx1100) row to the README/DETAILS rates tables, per the maintainer's request to split the rates row out of #228 and re-measure.
What it adds (docs only, current head):
| **IQ3_S** (AMD RX 7900 XTX) | 60 tokens/s | 1,641 tokens/s |760 | 1,275 | 1,641 | 1,594 | 1,494 | -65.1 | 60.4 | 64.4 | 61.8 | 59.2 | -Provenance (updated for the current head). Engine 0.1.38 on a ROCm 10.2 nightly with the tuned gfx1100 hipBLASLt-100500 table (upstream as #755), median of 3 non-stalled cells per tier from a 1K-128K one-shot sweep (256 generated tokens, greedy; Ryzen 9 7900X, 96 GB). Default configuration — no WMMA opt-ins (see the thread). 262K stays
-, same as the existing IQ3_S/IQ3_XXS rows. The flat decode is expected: GDN linear attention makes decode O(1) in context. On-device kernel-shape autotuning (#744) was run on this machine and kept every shipped shape — the row's kernels are the shipped defaults.Stalls were recorded and excluded from the medians. Docs-only change; no code. (The first revision of this description carried the 0.1.31 packaged-ROCm numbers; the commits above supersede them.)
Prepared with an AI engineering agent (GLM-5.3 / Z.ai) under human direction; every number is from our own recorded runs.