[mmbeanz]

What this is

A community benchmark report from an unequal two-card rig, answering docs/TEST_REQUESTS.md row 1 (two NVIDIA cards of different sizes, --layer-split auto) and A/B-ing the two live 0.1.43 default debates (#2010/#1975 overlap, route tail-skip) on a split. Note: the overlap turned out to be disabled on layer splits, so only the tail-skip A/B tests a live feature here; the overlap arm is a null control (see finding 4 in #2085). Those threads measured single-GPU only.

Results-only: one new folder, bench/results/2026-10-11-community-rtx-5060ti-3060-split/, no engine changes. The branch is from tag v0.1.43 (7695859a), the exact commit the engine was built from (plus one local setup.py-only pip->uv commit; no engine code touched).

Hardware

"bertha" - Ryzen 9 7900X, 64 GB DDR5-6400, Linux (CachyOS, engine in systemd-nspawn), driver 615.71.09, CUDA 13.4.

• RTX 5060 Ti 16 GB on PCIe 5.0 x8 - measured H2D 28.2-28.7 GB/s
• RTX 3060 12 GB on PCIe 3.0 x4 behind the chipset - measured H2D 3.15-3.2 GB/s

Links and H2D measured under load with tools/h2d_bench. The auto-generated machine block is unusable on this host (#2062) and was hand-corrected; the original values and every other edit are listed in NOTES.md.

Model

Qwen3.8-Flash-Next IQ3_S (ISTA-DASLab GSQ-RCO pack, sha256 in sha256sums.txt), 512K context (yarn 2.0), pinned expert profile - no --expert-profile-save anywhere.

Configurations tested

All tools/strata_bench.py --long --runs 3, one continuous evening session, 0 failures, /metrics?requests=all per-request histories included.

run	config
A	baseline --layer-split 40
B1	--layer-split auto (planner chose K=3)
B1c	explicit --layer-split 6 (the engine's printed suggestion, followed literally)
B1d	explicit --layer-split 42 (the as-given-frame equivalent)
C	baseline + STRATA_ADAPT_OVERLAP=0 (null control: overlap is already off on a split)
D	baseline + STRATA_ROUTE_TAIL_SKIP=0
needles	needle_bench.py 32K/128K/256K x depths 10/50/90: 9/9 found

Headline numbers

• Baseline: 32K prompt 1,721 tok/s, short-chat decode 74.4 tok/s.
• auto costs -25.0% of 32K prompt throughput (chunk clamp, Auto layer assigns only 1 layer to card dropping prefill due to prompt-chunk clamp #1616).
• The literal printed suggestion costs -76%: explicit splits do not inherit the Multi-GPU performance is ~2x lower than Francesco Albano fork despite the changes being merged into v0.1.40 #1352 card reorder - a question for maintainers is in the report.
• Tail-skip measures +5.2% here, not +10-20%, and decays with context; ADAPT_OVERLAP is inert on layer splits - the engine disables it before reading the flag (corrected after review in 0.1.43 on Linux CUDA: STRATA_ADAPT_OVERLAP costs 21% decode (single GPU, hybrid CPU, native expert pack) #2010; see commit 79cc236).
• Hit rate tracks per-layer coverage, not held-expert count (8,842 vs 5,935 held, statistically identical hit rate).

Not tested

Long soak, batching/concurrency, vision latency, mmap file tier, Windows/HIP, single-card configs on this box. hit_rate is the decode-phase metric only.

Discussion

The announcement issue with the full findings: #2085. The C and D rows were also posted as comments on #2010 and #2007; the #2010 comment's overlap conclusion was corrected afterwards (see the correction comment in #2010 and finding 4 in #2085).

Drafted with AI assistance; I reviewed every line and ran the measurements myself.

[Niko1221]

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.