Skip to content

setup: Volta (V100, sm_70) and Turing as an experimental build path - #130

Closed
DingoOz wants to merge 1 commit into
Niko1221:mainfrom
DingoOz:v100-sm70
Closed

DingoOz wants to merge 1 commit into
Niko1221:mainfrom
DingoOz:v100-sm70

Conversation

@DingoOz

@DingoOz DingoOz commented Sep 29, 2026

Copy link
Copy Markdown

Lets setup install Strata on a compute capability 7.x card (Tesla V100, RTX 20) instead of refusing it. The engine's kernels already have pre-sm_80 paths behind STRATA_EXPERIMENTAL_SM75; this makes setup use them.

Changes (setup.py)

  • GPU check: 7.x cards are accepted (below 7.0 is still refused), with a warning that this is experimental and the engine is compiled locally. The ready-made engine has no sm_70/75 code, so the existing "compiling instead" path takes over.
  • Build flag: -DSTRATA_EXPERIMENTAL_SM75=ON is passed when any chosen card is below sm_80.
  • CUDA toolkit: CUDA 13 dropped Volta. With an sm_70 card, find_nvcc picks the newest 12.x toolkit even if a 13.x is also installed, and the automatic install gets 12.9 instead of 13.0.
  • Host compiler (Linux): CUDA 12.x refuses a distribution's current GCC (for example CUDA 12.4 with GCC 15). When the default g++ is newer than the toolkit accepts, setup uses the newest gcc-N/g++-N it does accept for C, C++ and nvcc together. Setting only CMAKE_CUDA_HOST_COMPILER isn't enough: nvcc adds its GCC's library folder to the link line, so C++ objects built by GCC 15 then fail to link against GCC 12's libstdc++ (undefined reference to ...basic_string::_M_replace_cold). An existing CUDAHOSTCXX is left alone.

README.md gets a short note in "You need".

Tested

Tesla V100-PCIE-16GB, driver 580.178, CUDA 12.4, Ubuntu with GCC 15 as the default and GCC 12 installed:

  • engine 0.1.22 (this branch's main) builds for sm_70 through build_engine;
  • it serves Qwen3.8-Flash-Next IQ2_XS: correct answers, MTP drafts accepted, about 17-24 tok/s decode on short prompts;
  • a P100 (sm_60) in the same machine is still reported as not supported.

Not tested: Turing, Windows, the automatic CUDA 12.9 install.

A compute capability 7.x card is no longer refused: setup compiles the engine for it with
STRATA_EXPERIMENTAL_SM75 (the kernels already take their pre-sm_80 paths), and warns that it is
experimental. CUDA 13 dropped Volta, so with a sm_70 card the newest 12.x toolkit is used (and 12.9 is
the one installed when none is present). When the default GCC is newer than the toolkit accepts (CUDA
12.4 with GCC 15), a matching gcc-N/g++-N is used for C, C++ and nvcc together: nvcc puts its GCC's
library folder on the link line, so mixing versions fails to link against the older libstdc++.

Tested on a Tesla V100-PCIE-16GB (CUDA 12.4, GCC 15 default + GCC 12): engine 0.1.22 builds for sm_70
and serves Qwen3.8-Flash-Next IQ2_XS.
@Niko1221

Copy link
Copy Markdown
Owner

Thanks @DingoOz, the toolkit and host-compiler handling is really useful. Since 0.1.25 setup asks NVIDIA vs AMD on mixed PCs; please rebase, keep AMD as the default when the only NVIDIA cards are experimental (and don't mark them "recommended"), and confirm 0.1.26 still builds and answers on the V100. The architecture floor itself is being settled in #87.

@Niko1221 Niko1221 mentioned this pull request Sep 30, 2026
@Niko1221

Copy link
Copy Markdown
Owner

Thanks @DingoOz. Since 0.1.27 Turing (sm_75) is supported natively, so only Volta remains here. A new PR on current main would need: sm_70 accepted by the engine (the CMake and runtime floor), setup keeping AMD as the default on mixed PCs when the only NVIDIA cards are experimental, the CUDA 12.x/GCC selection aligned with the toolkit handling in #136, and a confirmation that it builds and answers on the V100. We're closing PRs that wait on changes so the list stays manageable; this is not a no. Please open a new PR when it's ready and link this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants