[Bug]: DeepSeek-V4.1 vision tower fails on SM107: cuDNN SDPA "No valid execution plans built" during KV-cache profiling #20586

System Info

• GPU: Vera Rubin, compute capability 10.7
• TensorRT-LLM main (tensorrt_llm/_torch/models/deepseek_v41_vision.py unchanged up to 0372799), PyTorch backend
• torch 2.14.0+cu132 with its bundled cuDNN 9.24.0, NVRTC 13.4.92
• Model: deepseek-ai/DeepSeek-V4.1-Flash (DeepseekV41ForConditionalGeneration, vision encoder enabled), TP4/EP4

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Reproduction

trtllm-serve on the model above, with the multimodal encoder enabled, dies on every rank while estimating KV-cache capacity, in the dummy image encode:

_util.py configure_kv_cache_capacity -> _encode_dummy_inputs
modeling_deepseekv41_multimodal.py encode_multimodal_inputs
deepseek_v41_vision.py -> F.scaled_dot_product_attention
RuntimeError: cuDNN Frontend error: [cudnn_frontend] Error: No valid execution plans built.

Standalone (1 GPU, compute capability 10.7, torch 2.14 + cuDNN 9.24), the same shape the vision attention uses for the 8192-patch profiling dummy:

import torch, torch.nn.functional as F
x = torch.randn(8192, 3 * 1024, device="cuda", dtype=torch.bfloat16)
q, k, v = (t.reshape(8192, 16, 64).transpose(0, 1).unsqueeze(0) for t in x.chunk(3, -1))
F.scaled_dot_product_attention(q, k, v)  # RuntimeError: ... No valid execution plans built.

Expected behavior

The server starts and image requests are served, as on SM100/SM103.

actual behavior

Startup fails as above. Only disable_mm_encoder: true gets the server up (text only).

additional notes

Root cause:

• PyTorch 2.14 (CUDA >= 13 builds) prefers the cuDNN SDPA backend on every SM 10.x device, including 10.7, and does not fall back to another backend when cuDNN plan building fails.
• cuDNN 9.24 offers only runtime-compiled engines (eng8/eng10) for SM 10.7 SDPA forward, and all of them fail NVRTC compilation (CUDNN_STATUS_INTERNAL_ERROR_COMPILATION_FAILED). This holds for every shape that dispatches to cuDNN (head_dim 32/64/128, BF16/FP16, contiguous or strided), with NVRTC 13.2 or 13.4. cuDNN 9.25.1.1, 9.26.1.1 and 9.27.0.42 provide an engine (eng18) that builds on SM 10.7.
• A try/except fallback in the model is not safe: after a failed build PyTorch caches a null graph for that shape and the next call on the same shape segfaults. Reported upstream as [cuDNN][SDPA] SM 10.7 + cuDNN < 9.25: cuDNN is preferred but no forward plan builds; retrying the same shape segfaults (null MHAGraphCache entry) pytorch/pytorch#201420.
• The vision tower's unit tests are CPU-only, and the V4.1 GPU tests (added with [TRTLLM-16621][feat] Support DeepSeek-V4.1-Flash #19727) are text-only and do not run on SM 10.7, so CI does not cover this path.

Vision tower forward on one Vera Rubin GPU (full 32-layer tower + aligner, real weights, BF16; 1022x1022 image = 5329 patches / 8192-patch profiling dummy):

Against an FP32 reference (the checkpoint's inference/vision.py), every BF16 backend has a relative L2 error of 3.5–3.8% at the aligner output (MATH in BF16: 3.7%), so the choice of backend is within BF16 noise.

Proposed fix: on SM 10.7 with cuDNN < 9.25, exclude only the cuDNN backend around the vision SDPA call; leave every other configuration unchanged, since cuDNN is the fastest backend where it works. PR to follow.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.