[lucifer1004]

Description

Fixes #20586.

The DeepSeek-V4.1 vision tower calls F.scaled_dot_product_attention. PyTorch (CUDA >= 13 builds) prefers cuDNN SDPA on SM 10.x. On SM 10.7 with cuDNN < 9.25 (e.g. torch 2.14.0+cu132 with its bundled cuDNN 9.24), cuDNN cannot build any forward plan, and PyTorch raises instead of falling back. This fails server startup in KV-cache profiling, which encodes a dummy image.

• _vision_sdpa_context(device) (new): on a CUDA device with compute capability 10.7 and cuDNN < 9.25 (_excludes_cudnn_sdpa, cached per device), the vision SDPA call runs under sdpa_kernel with only the FLASH / EFFICIENT / MATH backends that the caller has enabled; cuDNN is the only backend removed, and in practice FLASH is selected. Every other configuration (CPU, SM100/SM103, cuDNN >= 9.25) gets a nullcontext and is unchanged. The scope is kept narrow because cuDNN is the fastest backend wherever it works (on Vera Rubin with cuDNN 9.27 the tower is 26–33% faster than with FLASH).
• A try/except fallback is not used on purpose: after a failed build PyTorch caches a null graph for that shape, and the next call on the same shape segfaults ([cuDNN][SDPA] SM 10.7 + cuDNN < 9.25: cuDNN is preferred but no forward plan builds; retrying the same shape segfaults (null MHAGraphCache entry) pytorch/pytorch#201420).

Vision tower forward on one Vera Rubin GPU (compute capability 10.7; full 32-layer tower + aligner, real weights, BF16; 5329 / 8192 patches): cuDNN 9.24 fails; FLASH 30.8 / 53.0 ms; EFFICIENT 44.6 / 86.3 ms; MATH 260 / 567 ms; cuDNN 9.27 22.8 / 35.5 ms. Against an FP32 reference, every BF16 backend has a relative L2 error of 3.5–3.8% at the aligner output, so the backend choice is within BF16 noise.

Test Coverage

• tests/unittest/_torch/modeling/test_deepseek_v41_multimodal.py (CPU-only, unittest/_torch/modeling in l0_cpu): test_vision_sdpa_skips_cudnn_only_on_sm107_with_old_cudnn (capability and cuDNN version mocked: only (10, 7) with cuDNN 9.24 removes cuDNN; 9.25, (10, 0), (10, 3) and CPU leave the SDPA flags unchanged; flags are restored on exit) and test_vision_sdpa_keeps_backends_the_caller_disabled (a backend disabled by the caller stays disabled).
• tests/unittest/_torch/modeling/test_deepseek_v41_vision_sm107.py (new, GPU, skipped unless compute capability 10.7; added to l0_gr100.yml): one full-width vision block on a 32x32 grid with the default SDPA dispatch, compared with the MATH backend.
• Results on Vera Rubin (compute capability 10.7), torch 2.14.0+cu132 / cuDNN 9.24, same diff on a recent main: the SM 10.7 GPU test fails without the fix (the same cuDNN error) and passes with it; the existing tests in test_deepseek_v41_multimodal.py (4) pass with and without the fix; the full tower with the deepseek-ai/DeepSeek-V4.1-Flash weights runs with the fix, and its output is bitwise identical to forcing FLASH. Re-run on this commit (80828b1): the SM 10.7 GPU test fails without the fix and passes with it; test_deepseek_v41_multimodal.py passes 10/10 with the fix; the CPU gate cases (-k vision_sdpa) pass 6/6. End to end, trtllm-serve (TP4/EP4, vision encoder enabled) starts, passes our image test cases, and keeps text accuracy (GSM8K 98–99/100 vs 98 with the encoder disabled).
• Locally (CPU, torch 2.14.1): the new gate tests pass (6 cases); each of three mutations of the gate (threshold 9.26, capability check >= (10, 0), no filtering by the caller's flags) makes at least one of them fail. py_compile and pre-commit on the changed files: clean.

PR Checklist

• PR description clearly explains what and why.
• PR follows the TRT-LLM coding guidelines to the best of my knowledge.
• Test cases are provided for new code paths.
• No API changes, no new dependencies, no CODEOWNERS changes.
• Please check this after reviewing the above items as appropriate for this PR.