Skip to content

bench/results: gfx906 IQ3_XXS (2x Instinct MI50 / Radeon VII), and the same run with --mmap-experts - #1978

Closed
xxDoman wants to merge 1 commit into
Niko1221:mainfrom
xxDoman:bench-results-gfx906-iq3
Closed

xxDoman wants to merge 1 commit into
Niko1221:mainfrom
xxDoman:bench-results-gfx906-iq3

Conversation

@xxDoman

@xxDoman xxDoman commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

Two folders from one machine, deliberately one flag apart — same box, same model, same config:

  • resident expert arena (no --mmap-experts): bench/results/2026-10-11-community-2x-instinct-mi50-32gb-qwen3-8-flash-next-uncensored-iq3-xxs/
  • arena read through the page cache (--mmap-experts): bench/results/2026-10-11-community-2x-instinct-mi50-32gb-qwen3-8-flash-next-uncensored-iq3-xxs-low-ram/

Both are added to the list in docs/COMMUNITY_BENCHMARKS.md as well.

Hardware and model. AMD Instinct MI50 32 GB + Radeon VII 16 GB (gfx906), Core i5-12400F, 61.6 GiB RAM.
Qwen3.8-Flash-Next-Uncensored IQ3_XXS (pack orca-iq3_xxs, profile expert-profile.bin), context 131,072,
KV q4_0, prefill 2048, MTP spec 4, --layer-split auto, no --resident-* flags.

How it was run. tools/strata_bench.py --url http://127.0.0.1:8085 --runs 3 --clients 1, against a server
that was already running — hence no engine-log excerpt in the folders.

What the one flag is worth on a model that fits in RAM (medians over 3 runs):

resident --mmap-experts
4K prompt tok/s 546.9 (546.5-549.5) 473.6 (375.1-477.0) −13.4%
16K prompt tok/s 621.8 (619.7-622.0) 543.9 (543.2-546.1) −12.5%
4K / 16K first token 7.38 / 25.87 s 8.52 / 29.56 s +14-15%
chat decode 44.0 (41.9-62.1) 41.2 (34.8-53.0) ranges overlap
agent decode 41.7 (34.9-62.0) 44.7 (33.7-53.2) ranges overlap
1 concurrent client 31.7 tok/s 32.5 tok/s same

On this model the page-cache mode costs the prompt path about 12-13% and the first token 14-15%; decode is
not attributable at n=3. The wide range in the mmap arm's 4K row is one run paying for a cold page cache
(375.1). Not measured: the case low-RAM exists for — an arena larger than RAM, where the resident mode cannot
run at all.

Provenance — these are not stock 0.1.43 numbers. They come from the gfx906 image we maintain
(xxdoman/strata-mi50, engine built from tag v0.1.43). That build carries three HIP build gates (without
them the 0.1.40+ tree does not compile on gfx906) and the rocBLAS solution-table reader — the idea is in
#981, which is gated behind a path gfx906 never compiles, so we widened the gate. The image enables that
table together with the FP16 prompt path it needs (STRATA_HIP_PROMPT_F16=1, which the engine calls "not
bit-identical to the default"). The table is a prefill lever, not a decode one: measured by environment
switch on the same engine, +9.5% / +10.8% prompt at 16K / 32K, decode unchanged. Model files, packs and the
harness are the stock ones.

If it is more useful as a single folder, say which one and I will drop the other.

@Niko1221

Copy link
Copy Markdown
Owner

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

  1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
  2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
  3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
  4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.

@Niko1221 Niko1221 closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants