bug: version >=v3.15.0 not support CUDA v13.1 & v13.2 #577

Activity

1. xinxi053

   added

   bugSomething isn't workingSomething isn't working

   requires triageRequires triagingRequires triaging

   on Mar 10, 2026
2. xinxi053

   added the

   Bug

   issue type on Mar 10, 2026
3. giladgd commented on Mar 12, 2026

   

   giladgd

   on Mar 12, 2026

   Member

   More actions

   @xinxi053 Do you happen to have multiple CUDA versions installed?
   Also, have you updated the Nvidia driver to the latest version?
   Can you please try running npx --yes node-llama-cpp@latest and see whether it's able to detect the GPU?
   There have been a few changes in llama.cpp since 3.15.0 was shipped so the latest version might help with these.

   Also, if you build from source using the command below, does it compile successfully and the GPU is detected afterwards?

   npx --no node-llama-cpp source download

   I'm trying to figure out whether this is an issue that stems from the prebuilt binaries build of node-llama-cpp or something in llama.cpp itself.
   Since it seems that Vulkan also isn't detected, I suspect the issue might have something to do in llama.cpp, since I haven't changed anything related to Vulkan in 3.15.0.
   In 3.15.0 I had to update the CUDA version used in the CI due to changes in llama.cpp that use new features present only in CUDA 13.1, but that shouldn't have affected the Vulkan backend.
4. xinxi053 commented on Mar 13, 2026

   

   xinxi053

   on Mar 13, 2026

   Author

   More actions

   @giladgd
   1.Only CUDA 13.2 is installed in the current system.
   2.Has the Nvidia driver been updated to the latest version?
   Yes, it has been updated to the latest version.
   3.Executing the command: npx --yes node-llama-cpp@latest inspect gpu
   OS: Windows 10.0.26200 (x64) Node: 24.13.0 (x64) node-llama-cpp: 3.17.1 Prebuilt binaries: b8179 CUDA: CUDA is detected, but using it failed To resolve errors related to CUDA, see the CUDA guide: https://node-llama-cpp.withcat.ai/guide/CUDA Vulkan: Vulkan is detected, but using it failed To resolve errors related to Vulkan, see the Vulkan guide: https://node-llama-cpp.withcat.ai/guide/vulkan CPU model: 13th Gen Intel(R) Core(TM) i9-13900K Math cores: 0 Used RAM: 56.33% (17.88GB/31.75GB) Free RAM: 43.66% (13.86GB/31.75GB) Used swap: 60.38% (24.3GB/40.25GB) Max swap size: 40.25GB mmap: supported
   This command results in "CUDA&Vulkan failed" for node-llama-cpp@3.15.0 and all subsequent versions. For node-llama-cpp@3.14.5 and earlier versions, everything works normally, and both CUDA and Vulkan can be detected.
   4.Executing the command: npx --no node-llama-cpp source download --gpu vulkan
   The source code compiles successfully. Now I can support version 3.16.2.
   PS C:\Users\Administrator> npx --yes node-llama-cpp inspect gpu OS: Windows 10.0.26200 (x64) Node: 24.13.0 (x64) node-llama-cpp: 3.16.2 Prebuilt binaries: b8121 Cloned source: b8121 CUDA: available Vulkan: available CUDA device: NVIDIA GeForce RTX 4090 CUDA used VRAM: 6.21% (1.49GB/23.99GB) CUDA free VRAM: 93.78% (22.5GB/23.99GB) Vulkan device: NVIDIA GeForce RTX 4090 Vulkan used VRAM: 4.56% (1.08GB/23.57GB) Vulkan free VRAM: 95.43% (22.5GB/23.57GB) CPU model: 13th Gen Intel(R) Core(TM) i9-13900K Math cores: 24 Used RAM: 50.9% (16.16GB/31.75GB) Free RAM: 49.09% (15.59GB/31.75GB) Used swap: 71.63% (28.83GB/40.25GB) Max swap size: 40.25GB mmap: supported
   Final Issue:
   The problem lies with the prebuilt binary packages of node-llama-cpp. Verification is as follows:
   When executing the command: npm install -g node-llama-cpp@3.14.5 (installing the prebuilt binary package of 3.14.5), the directory
   C:\Users\Administrator\AppData\Roaming\npm\node_modules\node-llama-cpp\node_modules@node-llama-cpp
   contains the following folders: win-arm64, win-x64, win-x64-cuda, win-x64-cuda-ext, win-x64-vulkan.
   When executing the command: npm install -g node-llama-cpp@3.15.0, the directory only contains two folders: win-arm64 and win-x64.
   In other words, starting from version 3.15.0, the prebuilt binary packages no longer include support for CUDA and Vulkan.
5. giladgd commented on Mar 13, 2026

   

   giladgd

   on Mar 13, 2026

   Member

   More actions

   @xinxi053 Do you have a custom npm registry configured in your .npmrc or are you using the official one?
   Perhaps the prebuilt binary modules are missing from the npm registry that you use.

   You can run this command to see what npm registry you have configured:

   npm config get registry
6. xinxi053 commented on Mar 14, 2026

   

   xinxi053

   on Mar 14, 2026 via email

   Author

   More actions

   PS C:\Users\Administrator> npm config get registry https://registry.npmmirror.com Jon Snow ***@***.***
7. giladgd commented on Mar 14, 2026

   

   giladgd

   on Mar 14, 2026

   Member

   More actions

   @xinxi053 It seems that there's a size limit to the packages that this mirror can pull.
   I've opened a PR to fix it on their end.
   I'll also add an indication in the inspect gpu command for when prebuilt binary modules are missing, and when that happens also check for whether a custom npm registry is configured and log it as well. This should help detecting these kinds of issues faster.
8. giladgd

   removed

   requires triageRequires triagingRequires triaging

   on Mar 14, 2026
9. giladgd

   self-assigned this

   on Mar 14, 2026
10. github-merge-queue

    added a commit that references this issue on Mar 14, 2026

    feat: add `@node-llama-cpp` to `allowLargeScopes` (#518)

    ...

    d9b4ae5
11. giladgd commented on Mar 14, 2026

    

    giladgd

    on Mar 14, 2026

    Member

    More actions

    @xinxi053 This npm mirror should now have the latest prebuilt binary modules.
    Please try removing the node_modules directory and package-lock.json file, reinstall node-llama-cpp and the run the npx --yes node-llama-cpp inspect gpu command again and let me know whether it's now able to detect the GPU without building from source.
12. xinxi053 commented on Mar 15, 2026

    

    xinxi053

    on Mar 15, 2026

    Author

    More actions

    @giladgd
    Everything is working normally now.

    PS C:\Users\Administrator> npm install -g node-llama-cpp@3.16.2
    added 155 packages in 15s
    43 packages are looking for funding
    run npm fund for details

    C:\Users\Administrator\AppData\Roaming\npm\node_modules\node-llama-cpp\node_modules@node-llama-cpp
    Mode LastWriteTime Length Name

    ---

    d----- 2026/3/15/星期日 8:27 win-arm64
    d----- 2026/3/15/星期日 8:27 win-x64
    d----- 2026/3/15/星期日 8:27 win-x64-cuda
    d----- 2026/3/15/星期日 8:27 win-x64-cuda-ext
    d----- 2026/3/15/星期日 8:27 win-x64-vulkan

    PS C:\Users\Administrator> npx --yes node-llama-cpp inspect gpu
    OS: Windows 10.0.26200 (x64)
    Node: 24.13.0 (x64)
    node-llama-cpp: 3.16.2
    Prebuilt binaries: b8121
    CUDA: available
    Vulkan: available
    CUDA device: NVIDIA GeForce RTX 4090
    CUDA used VRAM: 6.21% (1.49GB/23.99GB)
    CUDA free VRAM: 93.78% (22.5GB/23.99GB)
    Vulkan device: NVIDIA GeForce RTX 4090
    Vulkan used VRAM: 4.56% (1.08GB/23.57GB)
    Vulkan free VRAM: 95.43% (22.5GB/23.57GB)
    CPU model: 13th Gen Intel(R) Core(TM) i9-13900K
    Math cores: 0
    Used RAM: 50.47% (16.02GB/31.75GB)
    Free RAM: 49.52% (15.72GB/31.75GB)
    Used swap: 53.68% (21.61GB/40.25GB)
    Max swap size: 40.25GB
    mmap: supported
13. giladgd commented on Mar 15, 2026

    

    giladgd

    on Mar 15, 2026

    Member

    More actions

    @xinxi053 Thank you for helping me solve this issue!
    I’ll release the new version with the better indications in the inspect gpu command soon.
14. giladgd

    closed this as completedin #573on Mar 15, 2026
15. github-actions commented on Mar 15, 2026

    

    github-actionsBot

    on Mar 15, 2026 – with GitHub Actions

    More actions

    🎉 This issue has been resolved in version 3.18.0 🎉

    The release is available on:

    • v3.18.0
    • npm package (@latest dist-tag)
    • GitHub release
    • v3.18.0

    Your semantic-release bot 📦🚀
16. github-actions

    added

    released

    on Mar 15, 2026