Skip to content

Misc. bug: Updating to V0.7.0 causes memory issue #31319

New issue
New issue
Closed
Closed
Misc. bug: Updating to V0.7.0 causes memory issue#31319
Labels
bug-unconfirmed

Description

@WSINTRA
WSINTRA
opened on Oct 28, 2026

Name and Version

version: 0.7.0-dev (build 11941, commit 19ff817)
built with AppleClang 21.0.0.21000101 for Darwin aarch64

Operating systems

No response

Which llama.cpp modules do you know to be affected?

No response

Command line

llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL --temp 1.0 \
    --reasoning-preserve \
    --top-p 0.95 \
    --top-k 20 \
    --min-p 0.00 \
    --spec-type draft-mtp \
    --chat-template-kwargs '{"preserve_thinking": true, "reasoning_effort": "medium"}' --cors-origins localhost

Problem description & steps to reproduce

Ran update from v0.5.0 (everything worked fine before update)
After update I now get memory errors. See log below

└> llama version
version: 0.5.0-dev (build 11379, commit 1537a0a)
built with AppleClang 21.0.0.21000101 for Darwin aarch64
┌[kiwish-4.2]-(~)-
└> llama update
Version: b11941
Probing Metal...
Found: m2
Downloading llama...
Installation completed successfully

Run the following command to start it:

llama serve

┌[kiwish-4.2]-(~)-
└> llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL --temp 1.0
--reasoning-preserve
--top-p 0.95
--top-k 20
--min-p 0.00
--spec-type draft-mtp
--chat-template-kwargs '{"preserve_thinking": true, "reasoning_effort": "medium"}' --cors-origins localhost

0.00.001.967 I srv llama_server: initializing ...
0.03.272.470 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the -lv N CLI arg)
0.03.276.006 I srv load_model: loading model 'unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL'
0.06.597.919 I cmn init: llama threadpool init, n_threads = 16
0.06.963.662 I common_speculative_init_result: creating MTP draft context against the target model '/Users/ws/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-UD-Q8_K_XL.gguf'
0.07.286.150 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
0.07.286.154 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
0.07.286.155 W load_hparams: more info: #16842

0.07.892.597 I srv load_model: loaded multimodal model, '/Users/ws/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/mmproj-BF16.gguf'
0.07.892.604 I srv load_model: initializing, n_slots = 4, n_ctx_slot = 257280, kv_unified = 'true'
0.08.144.034 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.08.144.043 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
0.08.150.027 I srv server_run_i: model loaded
0.08.150.032 I srv server_on_li: listening on http://127.0.0.1:9931
^C0.21.742.381 I srv operator(): operator(): cleaning up before exit...
0.21.743.735 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.21.743.750 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
0.21.744.181 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.21.744.190 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
┌[kiwish-4.2]-(~)-
└> llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL --temp 1.0
--reasoning-preserve
--top-p 0.95
--top-k 20
--min-p 0.00
--spec-type draft-mtp
--chat-template-kwargs '{"preserve_thinking": true, "reasoning_effort": "medium"}' --cors-origins localhost

0.00.001.032 I srv llama_server: initializing ...
0.00.487.848 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the -lv N CLI arg)
0.00.490.065 I srv load_model: loading model 'unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL'
0.04.268.170 I cmn init: llama threadpool init, n_threads = 16
0.04.471.728 I common_speculative_init_result: creating MTP draft context against the target model '/Users/ws/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-UD-Q8_K_XL.gguf'
0.04.595.863 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
0.04.595.867 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
0.04.595.868 W load_hparams: more info: #16842

0.05.052.684 I srv load_model: loaded multimodal model, '/Users/ws/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/mmproj-BF16.gguf'
0.05.052.712 I srv load_model: initializing, n_slots = 4, n_ctx_slot = 257280, kv_unified = 'true'
0.05.265.569 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.05.265.577 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
0.05.269.826 I srv server_run_i: model loaded
0.05.269.833 I srv server_on_li: listening on http://127.0.0.1:9931
^C0.08.708.161 I srv operator(): operator(): cleaning up before exit...
0.08.709.206 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.08.709.221 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
0.08.709.447 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.08.709.451 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
┌[kiwish-4.2]-(~)-
└> llama --version
version: 0.7.0-dev (build 11941, commit 19ff817)
built with AppleClang 21.0.0.21000101 for Darwin aarch64

First Bad Commit

No response

Relevant log output

Logs 
└> llama version
version: 0.5.0-dev (build 11379, commit 1537a0a8b)
built with AppleClang 21.0.0.21000101 for Darwin aarch64
┌[kiwish-4.2]-(~)-
└> llama update
Version: b11941
Probing Metal...
Found: m2
Downloading llama...
Installation completed successfully

Run the following command to start it:

  llama serve

┌[kiwish-4.2]-(~)-
└> llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL --temp 1.0 \
    --reasoning-preserve \
    --top-p 0.95 \
    --top-k 20 \
    --min-p 0.00 \
    --spec-type draft-mtp \
    --chat-template-kwargs '{"preserve_thinking": true, "reasoning_effort": "medium"}' --cors-origins localhost

0.00.001.967 I srv  llama_server: initializing ...
0.03.272.470 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.03.276.006 I srv    load_model: loading model 'unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL'
0.06.597.919 I cmn          init: llama threadpool init, n_threads = 16
0.06.963.662 I common_speculative_init_result: creating MTP draft context against the target model '/Users/ws/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-UD-Q8_K_XL.gguf'
0.07.286.150 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
0.07.286.154 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
0.07.286.155 W load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842

0.07.892.597 I srv    load_model: loaded multimodal model, '/Users/ws/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/mmproj-BF16.gguf'
0.07.892.604 I srv    load_model: initializing, n_slots = 4, n_ctx_slot = 257280, kv_unified = 'true'
0.08.144.034 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.08.144.043 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
0.08.150.027 I srv  server_run_i: model loaded
0.08.150.032 I srv  server_on_li: listening on http://127.0.0.1:9931
^C0.21.742.381 I srv    operator(): operator(): cleaning up before exit...
0.21.743.735 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.21.743.750 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
0.21.744.181 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.21.744.190 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
┌[kiwish-4.2]-(~)-
└> llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL --temp 1.0 \
    --reasoning-preserve \
    --top-p 0.95 \
    --top-k 20 \
    --min-p 0.00 \
    --spec-type draft-mtp \
    --chat-template-kwargs '{"preserve_thinking": true, "reasoning_effort": "medium"}' --cors-origins localhost

0.00.001.032 I srv  llama_server: initializing ...
0.00.487.848 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.490.065 I srv    load_model: loading model 'unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL'
0.04.268.170 I cmn          init: llama threadpool init, n_threads = 16
0.04.471.728 I common_speculative_init_result: creating MTP draft context against the target model '/Users/ws/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-UD-Q8_K_XL.gguf'
0.04.595.863 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
0.04.595.867 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
0.04.595.868 W load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842

0.05.052.684 I srv    load_model: loaded multimodal model, '/Users/ws/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/mmproj-BF16.gguf'
0.05.052.712 I srv    load_model: initializing, n_slots = 4, n_ctx_slot = 257280, kv_unified = 'true'
0.05.265.569 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.05.265.577 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
0.05.269.826 I srv  server_run_i: model loaded
0.05.269.833 I srv  server_on_li: listening on http://127.0.0.1:9931
^C0.08.708.161 I srv    operator(): operator(): cleaning up before exit...
0.08.709.206 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.08.709.221 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
0.08.709.447 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
0.08.709.451 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)
┌[kiwish-4.2]-(~)-
└> llama --version
version: 0.7.0-dev (build 11941, commit 19ff817fb)
built with AppleClang 21.0.0.21000101 for Darwin aarch64

Activity

  1. WSINTRA
    added
    bug-unconfirmed
    on Oct 28, 2026

  2. WSINTRA commented on Oct 28, 2026

    @WSINTRA
    WSINTRA
    on Oct 28, 2026
    Author

    Tried the old, reboot to see if that fixes an issue but still same.
    0.38.667.834 E ggml_metal_synchronize: error: command buffer 1 failed with status 5
    0.38.667.840 E error: Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)

  3. WSINTRA commented on Oct 28, 2026

    @WSINTRA
    WSINTRA
    on Oct 28, 2026
    Author

    Tried with another model..
    llama serve -hf unsloth/flint-GGUF:Q8_0

    No memory issues so far, so issue was specific to the following command.
    llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL --temp 1.0
    --reasoning-preserve
    --top-p 0.95
    --top-k 20
    --min-p 0.00
    --spec-type draft-mtp
    --chat-template-kwargs '{"preserve_thinking": true, "reasoning_effort": "medium"}' --cors-origins localhost

  4. WSINTRA commented on Oct 28, 2026

    @WSINTRA
    WSINTRA
    on Oct 28, 2026
    Author

    Tried the following and it worked without memory issues.

    ws@Mac:~/.cache/huggingface/hub|⇒ llama serve -hf unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL
    0.00.001.064 I srv llama_server: initializing ...
    0.00.459.124 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the -lv N CLI arg)
    0.00.459.426 W srv server_setup: security: no API key is set and CORS allows all origins (see #25655)
    0.00.462.726 I srv load_model: loading model 'unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL'
    0.00.772.977 W model has unused tensor blk.64.attn_norm.weight (size = 20480 bytes) -- ignoring
    0.00.772.985 W model has unused tensor blk.64.post_attention_norm.weight (size = 20480 bytes) -- ignoring
    0.00.772.992 W model has unused tensor blk.64.attn_q.weight (size = 125829120 bytes) -- ignoring
    0.00.772.995 W model has unused tensor blk.64.attn_k.weight (size = 10485760 bytes) -- ignoring
    0.00.772.997 W model has unused tensor blk.64.attn_v.weight (size = 10485760 bytes) -- ignoring
    0.00.773.005 W model has unused tensor blk.64.attn_output.weight (size = 33423360 bytes) -- ignoring
    0.00.773.007 W model has unused tensor blk.64.attn_q_norm.weight (size = 1024 bytes) -- ignoring
    0.00.773.009 W model has unused tensor blk.64.attn_k_norm.weight (size = 1024 bytes) -- ignoring
    0.00.773.011 W model has unused tensor blk.64.ffn_gate.weight (size = 94699520 bytes) -- ignoring
    0.00.773.014 W model has unused tensor blk.64.ffn_down.weight (size = 94699520 bytes) -- ignoring
    0.00.773.016 W model has unused tensor blk.64.ffn_up.weight (size = 94699520 bytes) -- ignoring
    0.00.773.020 W model has unused tensor blk.64.nextn.eh_proj.weight (size = 104857600 bytes) -- ignoring
    0.00.773.023 W model has unused tensor blk.64.nextn.enorm.weight (size = 20480 bytes) -- ignoring
    0.00.773.025 W model has unused tensor blk.64.nextn.hnorm.weight (size = 20480 bytes) -- ignoring
    0.00.773.033 W model has unused tensor blk.64.nextn.shared_head_norm.weight (size = 20480 bytes) -- ignoring
    0.16.363.386 I cmn init: llama threadpool init, n_threads = 16
    0.16.603.912 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
    0.16.603.919 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
    0.16.603.919 W load_hparams: more info: #16842

    0.17.048.528 I srv load_model: loaded multimodal model, '/Users/ws/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/mmproj-BF16.gguf'
    0.17.497.696 I srv load_model: initializing, n_slots = 4, n_ctx_slot = 262144, kv_unified = 'true'
    0.17.502.358 W srv init: chat template supports preserving reasoning, it is enabled by default (may use more tokens, disable via --no-reasoning-preserve)
    0.17.502.375 I srv server_run_i: model loaded
    0.17.502.382 I srv server_on_li: listening on http://127.0.0.1:9931

  5. WSINTRA commented on Oct 28, 2026

    @WSINTRA
    WSINTRA
    on Oct 28, 2026
    Author

    Deduction to the rescue.
    Once I removed --spec-type draft-mtp from the call the memory issue went away

  6. WSINTRA
    closed this as completedon Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug-unconfirmed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions