FAQ #84

Kept up to date by the maintainers. Check here before opening an issue. See also the Roadmap and CONTRIBUTING.md.

Requirements

Q: What hardware and OS are supported?

x86_64 only, with an NVIDIA GPU from Ampere (RTX 30 series) up and driver r580+ (CUDA 13). Windows and Linux through the Desktop app; Linux through pip. macOS, AMD, DGX Spark (aarch64) are on the Roadmap.

Q: How much RAM do I need?

For MoE models the experts live in host RAM, so you need roughly the size of the expert weights in free RAM. For example Qwen/Qwen3.6-35B-A3B (bf16) needs about 70 GB; the NVFP4 checkpoint of the same model needs far less.

Q: Which models are supported?

See models.md. Other checkpoints of the same architectures usually work too. Support for GGUF quantization types and architectures is on the Roadmap.

Troubleshooting

Q: On Linux a model fails to start with fatal error: Python.h: No such file or directory.

Triton compiles a small C helper the first time it runs, so it needs a C compiler and the Python development headers. The Python that ships with a Linux distribution may come without the headers. Install them and start the model again; no reinstall is needed: sudo apt install gcc python3-dev (Fedora: sudo dnf install gcc python3-devel).