Linux sm_120: engine built with CUDA 12.8 crashes in MTP prefill (illegal memory access); CUDA 13.0 build works #220

Environment: Ubuntu 24.04, RTX PRO 5000 Blackwell 48 GB (sm_120), driver 595.91, engine 0.1.27 (a790805), IQ3_S, --max-context 262144, --vision gpu, 250 GB RAM, CPU without AVX-512.

There is no Linux prebuilt (every release ships only strata-windows-x64.zip), so setup compiled the engine. The host toolkit was CUDA 12.8, which install_build_tools accepts for sm_120 (need_cuda = (12, 8), setup.py:951), so setup built with it without asking for 13.0.

Symptom: a short request works (28-token prompt, 117 tok/s). Any prompt of a few thousand tokens kills the engine:

prefill copy_i32: an illegal memory access was encountered

That is the MTP drafter's prefill capture (src/core/mtp.cpp:516, copy_i32_from_mapped). It is deterministic. Things that did NOT change it:

• --prefill 2048 (auto had picked 8192 on the 48 GB card)
• --max-context 131072
• --vram-reserve-mib 700 -> 1200 (auto sizing had left 194 MiB free, see 0.1.27 on 16 GB + vision: the CJK draft leaves 156 MiB VRAM free (below the 256 MiB stall threshold); setup's fixed reserve 700 no longer suffices #199)

The same model and version run fine on Windows with the release build (CUDA 13.0).

Fix that worked: build the same source with CUDA 13.0 (inside nvidia/cuda:13.0.2-devel-ubuntu24.04, same cmake defs as setup.py:1076-1085), and use the CUDA_WHEELS from setup.py for the runtime. Now a 17,104-token prompt prefills at 2,473 tok/s and decodes at 98 tok/s.

Two smaller things we hit on the way:

1. lib_dirs from the run config is applied to the engine's environment only (serve/server.py:499), not to strata-vision. A vision helper built against CUDA 13 then fails with the vision encoder did not start because it can't find libcudart.so.13, libcublas.so.13 or libnccl.so.2. We worked around it with LD_LIBRARY_PATH in our launcher, plus pip install nvidia-nccl-cu13.
2. The generated run-*.sh doesn't keep --gpu unless you also pass --setup. On a two-GPU box that means the engine can land on the wrong card.

Suggestions: require 13.0 on Linux sm_120 (or warn loudly when building with 12.x), or publish a strata-linux-x64.zip.

Thanks for the engine, the numbers are impressive.