Repository navigation
Latest build also supports Volta #236
Description
I have a dual Xeon server with a 32GB V100, and have been experimenting with local LLMs since a few weeks.
Last week I saw your Reddit post and tried to install Strata even though sm_70 was officially not supported - it did not work.
This morning I saw a mention of Strata on the ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF model card, and when I noticed that your latest build now also supported sm_75, I once again tried to install and now it works correctly.
Only thing that needed changing were the artificial blocks in CMakeLists, device.cu and setup.py.
I only tested with swift-1.5-iq3_xxs and coder-IQ1_M.
Activity
Psxmaster83 commentedon Sep 30, 2026
nice ^^
wdlq commentedon Sep 30, 2026
With hardware like yours, is it possible to run the swift-1.5-iq3_xxs model with the full 256K context window?
MahirZukic commentedon Sep 30, 2026
What is this dashboard you are using? It looks nice and clean.
cardonja commentedon Sep 30, 2026
It is, but with 220 less experts in VRAM, and tg seems to settle at about 45 tps with 50K context used.
cardonja commentedon Sep 30, 2026
The default Strata dashboard:
Niko1221 commentedon Sep 30, 2026
Thanks for testing! sm_70 stays community-tested: 0.1.31 lets the existing experimental build flag (
-DSTRATA_EXPERIMENTAL_SM60=ON) also admit Volta, and the engine's startup check honours it (it refused anything below 7.5 even in that build). It won't be in the ready-made engine. Closing.Experimental Pascal/Volta build: admit sm_70, and the device check ho…
iq_avx2: build the IQ2 sign table without a variable shift
docs: NVIDIA_V100.md and the V100 prompt-speed measurements
docs: NVIDIA_V100.md and the measurements of the Volta prompt attention
iq_avx2: build the IQ2 sign table without a variable shift
docs: NVIDIA_V100.md and the measurements of the Volta prompt attention
Experimental Pascal/Volta build: admit sm_70, and the device check ho…
iq_avx2: build the IQ2 sign table without a variable shift
docs: NVIDIA_V100.md and the measurements of the Volta prompt attention
Experimental Pascal/Volta build: admit sm_70, and the device check ho…
iq_avx2: build the IQ2 sign table without a variable shift
docs: NVIDIA_V100.md and the measurements of the Volta prompt attention
Experimental Pascal/Volta build: admit sm_70, and the device check ho…
iq_avx2: build the IQ2 sign table without a variable shift
docs: NVIDIA_V100.md and the measurements of the Volta prompt attention
jiangrun0213 commentedon Oct 7, 2026
Volta (2× V100-SXM2-16GB) results: 125B MoE, v0.1.38 → v0.1.40 upgrade, and a config gotcha
Reporting Volta numbers as requested ("We now have a P100 for sm_60, but not Volta"). Thanks for keeping the experimental flag alive — it works great.
Hardware: 2× Tesla V100-SXM2-16GB (SXM2→PCIe adapters, NVLink NV6 active, measured 142.5 GB/s P2P), Xeon E5-2666 v3, X99 board with x8x8 bifurcation.
Model: Qwen3.8-Flash-Next-125B, IQ3_S (77.9 GiB GGUF → pack), int8 KV, 256K context, MTP (
--spec 4).Build: CUDA 12.8,
-DSTRATA_EXPERIMENTAL_SM60=ON -DCMAKE_CUDA_ARCHITECTURES=70, zero errors on v0.1.38 and v0.1.40.1. FetchContent short-circuited to a local llama.cpp worktree at the pinned commit. Also builtstrata-vision(CPU variant) — multimodal works.Decode (512-tok generations, server-side):
The v0.1.40 multi-GPU rework is a big win on this exact setup (+21%).
Prefill: 459 tok/s @ 3.7K prompt → 765 tok/s @ 14.5K (IQ3_S). MTP draft acceptance 70.5–71.5%. KV reuse: same 3.7K prompt re-sent = 8017 ms → 58 ms.
Config migration gotcha for anyone on 0.1.38:
--peer-deviceand--layer-splitare now mutually exclusive — the engine exits immediately if both are present. On 2 GPUs just uselayer_split: "auto"and drop all--peer-*flags; the helper-GPU expert caching is handled automatically now.Full notes, benchmarks and scripts: https://github.com/jiangrun0213/strata-v100-notes
Happy to test anything else on this box (it runs 24/7 as a systemd service).
iq_avx2: build the IQ2 sign table without a variable shift