Repository navigation
Conversation
…K, low-RAM mode Results-only submission from tools/strata_bench.py (suite 1, 3 runs per test). The same GPU as 2026-10-04-community-rtx-5070ti-windows and 2026-10-06-community-rtx5070ti-5900x, but on 31.6 GiB of system RAM instead of 64 GiB, so this install cannot keep the experts in RAM and runs the low-RAM mode (--mmap-experts) with them served through the OS file cache. Decode is about half and prompt processing 4-8x slower than the 64 GiB figures; that boundary is what the report measures. No engine changes: three files under bench/results/ only.
This was referenced Oct 18, 2026
[Benchmark] RTX 5070 12 GB + 32 GB RAM on Windows 10, Flash-Next IQ2_XS low-RAM mode (0.2.3.1)
#3063
Closed
Owner
|
Thanks for the pull request. We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected. To reopen it takes about two minutes:
The rules are in CONTRIBUTING.md. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a results-only report for an RTX 5070 Ti (16 GiB) with 31.6 GiB of system RAM — the same GPU as
2026-10-04-community-rtx-5070ti-windowsand2026-10-06-community-rtx5070ti-5900x, but at half their RAM, which inbench/results/is a combination that is not covered yet.Only three files under
bench/results/2026-10-17-community-rtx-5070-ti-qwen3-8-flash-next-iq2-xs/(README.md,results.json,engine-log-excerpt.txt). No engine or code changes, as the guide asks for a results-only submission.Hardware
Model and configuration
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF@ed59f92, IQ2_XS, both shards with their SHA-256q4_0, KV streaming off, low-RAM mode (--mmap-experts), expert cacheauto(~5,000 slots / ~6.8 GiB VRAM),--prefill auto, MTP--spec 4 --spec-min-p 0.5,--vram-reserve-mib 1600temperature 0Results (medians, ranges over 3 runs)
Total suite time 420 s.
What the report is about — and its limitation
This install cannot keep the 36 GB expert arena in 31.6 GiB of RAM, so unlike the two 64 GiB 5070 Ti reports it runs the low-RAM mode and serves the experts through the OS file cache. The measured boundary:
So on a 32 GiB machine the prompt/TTFT side, not decode, is what decides whether the setup is usable (a 16K prompt is 36 s before the first token). The chat row's 36.8-87.0 range is the low-RAM cache warming across the nine back-to-back requests, not run-to-run noise; the report says so in the Results section.
Also recorded because the guide asks for them: storage, PCIe link, power limit and the background load are filled in, and two things the tool got wrong are corrected — it reports the OS as "Windows 10 (10.0.26100)" (actually Windows 11 Pro 10.0.26200) and its own README template asks for facts it cannot read (CUDA version, commit on a zip install).
One tool bug found while preparing this
tools/strata_bench.py:564readsCMakeLists.txtwithread_text()and no encoding, so on a Chinese/Japanese Windows (GBK code page) it raisesUnicodeDecodeError: 'gbk' codec can't decode byte 0xbf in position 2on the file's UTF-8 BOM and exits before any measurement.setup.py:4649reads the same file withencoding="utf-8".PYTHONUTF8=1works around it. Happy to file it as an issue with the traceback if that is more useful than fixing it in this PR — I left it out of this PR to keep it results-only.