[ggerganov]

Overview

cont #23897
fix #25228 (comment)

Allow adding multiple iGPU devices of the same backend (e.g. CUDA). For example, this is useful when using virtual CUDA devices (#25228) on DGX Spark (treated as "integrated" GPU).

GGML_CUDA_DEVICES=2 ./bin/llama-completion -hf ggml-org/Qwen3-0.6B-GGUF:Q8_0 -p "I believe the meaning of life is" -n 32 --sampling-seq "k" --top-k 1 -no-cnv -lv 4

0.01.573.213 I common_memory_breakdown_print: | memory breakdown [MiB]                                 | total    free    self   model   context   compute    unaccounted |
0.01.573.215 I common_memory_breakdown_print: |   - CUDA0 (GB10 (physical device 0, virtual device 0)) | 61286 = 55454 + (2831 =   239 +    2400 +     192) +        3000 |
0.01.573.216 I common_memory_breakdown_print: |   - CUDA1 (GB10 (physical device 0, virtual device 1)) | 61286 = 55454 + (2919 =   364 +    2080 +     474) +        2912 |
0.01.573.216 I common_memory_breakdown_print: |   - Host   

CI run: https://github.com/ggml-org/llama.cpp/actions/runs/31581256852/job/94064620191

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO

[0cc4m]

Is it not possible to just set -dev for those virtual GPU CI cases? It sounds like an edge case that doesn't really warrant a code change for default device selection. But if that is not possible this PR is probably fine.

[ggerganov]

The goal is virtual devices to become widely used in the CI, so not really an edge case. In any case, it's not more of an edge case compared to the use case that introduced this workaround in #23897. The idea when specifying virtual devices is for the developers to get the same experience as if they had multiple GPUs on their machine, which does not require listing them with -dev normally.

[0cc4m]

I don't agree my original case was an edge case, it was just trying to use an Nvidia/AMD iGPU on a llama.cpp version compiled with CUDA/ROCm + Vulkan support.

But regardless, I can't think of a case where the logic of this PR would cause an issue, so it's probably fine.