Repository navigation
bench/results: gfx906 IQ3_XXS (2x Instinct MI50 / Radeon VII), and the same run with --mmap-experts - #1978
Closed
xxDoman wants to merge 1 commit into
Closed
bench/results: gfx906 IQ3_XXS (2x Instinct MI50 / Radeon VII), and the same run with --mmap-experts#1978xxDoman wants to merge 1 commit into
xxDoman wants to merge 1 commit into
Conversation
…e same run with --mmap-experts
Owner
|
Thanks for the pull request. We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected. To reopen it takes about two minutes:
The rules are in CONTRIBUTING.md. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two folders from one machine, deliberately one flag apart — same box, same model, same config:
--mmap-experts):bench/results/2026-10-11-community-2x-instinct-mi50-32gb-qwen3-8-flash-next-uncensored-iq3-xxs/--mmap-experts):bench/results/2026-10-11-community-2x-instinct-mi50-32gb-qwen3-8-flash-next-uncensored-iq3-xxs-low-ram/Both are added to the list in
docs/COMMUNITY_BENCHMARKS.mdas well.Hardware and model. AMD Instinct MI50 32 GB + Radeon VII 16 GB (gfx906), Core i5-12400F, 61.6 GiB RAM.
Qwen3.8-Flash-Next-Uncensored IQ3_XXS (pack
orca-iq3_xxs, profileexpert-profile.bin), context 131,072,KV q4_0, prefill 2048, MTP spec 4,
--layer-split auto, no--resident-*flags.How it was run.
tools/strata_bench.py --url http://127.0.0.1:8085 --runs 3 --clients 1, against a serverthat was already running — hence no engine-log excerpt in the folders.
What the one flag is worth on a model that fits in RAM (medians over 3 runs):
--mmap-expertsOn this model the page-cache mode costs the prompt path about 12-13% and the first token 14-15%; decode is
not attributable at n=3. The wide range in the mmap arm's 4K row is one run paying for a cold page cache
(375.1). Not measured: the case low-RAM exists for — an arena larger than RAM, where the resident mode cannot
run at all.
Provenance — these are not stock 0.1.43 numbers. They come from the gfx906 image we maintain
(
xxdoman/strata-mi50, engine built from tag v0.1.43). That build carries three HIP build gates (withoutthem the 0.1.40+ tree does not compile on gfx906) and the rocBLAS solution-table reader — the idea is in
#981, which is gated behind a path gfx906 never compiles, so we widened the gate. The image enables that
table together with the FP16 prompt path it needs (
STRATA_HIP_PROMPT_F16=1, which the engine calls "notbit-identical to the default"). The table is a prefill lever, not a decode one: measured by environment
switch on the same engine, +9.5% / +10.8% prompt at 16K / 32K, decode unchanged. Model files, packs and the
harness are the stock ones.
If it is more useful as a single folder, say which one and I will drop the other.