Multiple system prompts break Qwen3.5-* models

[DaGhostman]

Description

As part of investigating what could've caused Opencode-DCP/opencode-dynamic-context-pruning#407 I noticed that the tool was adding another system prompt, which caused for this model to break.

Despite being fixed in the plugin it makes sense to at least have it as a known issue, so this is mostly for documentation purposes and if there are other plugins that could incur such behavior for the qwen3.5 or other models.

Also maybe as a suggestion if there are multiple system prompts to be concatenated into a single one before dispatch to the model, similarly to how it is done here

Plugins

@tarquinen/opencode-dcp

OpenCode version

1.2.12

Steps to reproduce

1. Install @tarquinen/opencode-dcp with version <= 2.1.7
2. Start a session with any of the Qwen3.5 models
3. Send a message and expect to receive error this error

Screenshot and/or share link

No response

Operating System

EndeavourOS

Terminal

Ghostty

[marcobox]

I am noticing the same issue from my side as well. Here is my config:

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": [
    "opencode-websearch-cited@latest",
    "opencode-vibeguard@latest",
    "@franlol/opencode-md-table-formatter@latest"
  ],
  "provider": {
    "litellm": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "LiteLLM",
      "options": {
        "baseURL": "https://litellm.example.com/v1"
      },
      "models": {
        "gondor/qwen3-coder-next:80b": {
          "name": "Gondor Qwen3 Coder Next"
        },
        "llama-swap/qwen3.5:122b-q4": {
          "name": "Gondor Qwen3.5 Q4 Generic"
        }
      }
    },
    "google": {
      "options": {
        "websearch_cited": {
          "model": "gemini-2.5-flash"
        }
      },
      "whitelist": [
        "gemini-2.5-flash",
        "gemini-2.5-pro",
        "gemini-3-flash-preview",
        "gemini-3-pro-preview",
        "gemini-3.1-pro-preview"
      ]
    }
  }
}

Here is the an extract of what litellm sees:

{
  "issue_type": "litellm_internal_server_error",
  "litellm_version": "1.81.14",
  "model_group": "gondor/qwen3.5:122b-q4",
  "deployment": "openai/qwen3.5:122b-q4",
  "status": "failure",
  "error": {
    "class": "InternalServerError",
    "code": "500",
    "message": "Jinja Exception: System message must be at the beginning."
  },
  "request_shape": {
    "endpoint_path": "/v1/chat/completions",
    "stream": true,
    "tool_choice": "auto",
    "tools_count": 14,
    "messages_order": [
      {
        "role": "system",
        "content_preview": "system prompt 1..."
      },
      {
        "role": "system",
        "content_preview": "system prompt 2..."
      },
      {
        "role": "user",
        "content_preview": "test"
      }
    ]
  },
  "request_excerpt": {
    "model": "gondor/qwen3.5:122b-q4",
    "top_p": 1,
    "stream": true,
    "max_tokens": 32000,
    "tool_choice": "auto",
    "tools_count": 14,
    "messages": [
      {
        "role": "system",
        "content_preview": "You are opencode, an interactive CLI tool..."
      },
      {
        "role": "system",
        "content_preview": "<EXTREMELY_IMPORTANT> You have superpowers..."
      },
      {
        "role": "user",
        "content_preview": "test"
      }
    ]
  },
  "retry_context": {
    "max_retries": 2,
    "attempted_retries": 2
  },
  "timing": {
    "latency_seconds": 1.7045
  },
  "sanitized_environment": {
    "api_base": "***REDACTED_INTERNAL_URL***",
    "endpoint": "***REDACTED_INTERNAL_URL***",
    "hostnames": "***REDACTED***",
    "ip_addresses": "***REDACTED***",
    "api_keys_tokens": "***REDACTED***",
    "user_identifiers": "***REDACTED***"
  }
}

I guess in my case superpowers is causing the issue?

[DaGhostman]

yep, seems like it. I am not 100% sure what will be the correct approach, but I think that the plugin/mcp/etc adding those should be updated

[cyrilmhansen]

I put together a candidate fix in my fork:

cyrilmhansen#2

This patch normalizes the final system prompt into a single system message before provider request construction.

It is aimed at cases like the one reported here with Qwen, where multiple system prompt fragments can accumulate from provider/user/plugin input and strict chat templates appear to be sensitive to that final message shape.

[mingfang]

The problem is https://github.com/anomalyco/opencode/blob/dev/packages/opencode/src/session/llm.ts sends multiple system prompts and Qwen models(perhaps others) only supports one system prompt.
There's multiple open issues and PRs to fix this but don't understand what is the holdup.

[Milor123]

✅ Confirmed: The system prompt merge fix works perfectly (#23656

I just tested this PR locally and can confirm it completely resolves the Jinja Exception: System message must be at the beginning error.

My setup:

• Model: HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive (Q4_K_M)
• Server: llama-server with --jinja (required by this model's official template)
• Client: OpenCode (built from this PR branch)
• Context: 131072, tool calling enabled

With this patch, OpenCode correctly merges all system/developer messages into a single prompt before sending it to llama.cpp. The strict Jinja validation passes, and the model responds without breaking or hallucinating format.

🙏 Huge thanks to @zhangdw156 and the maintainers! This unblocks local Qwen3.x users who rely on strict chat templates.

❓ Quick question: Do you have an ETA for merging this into dev/main? Will it land in the next stable release (v1.14.x or v1.15.x)? I'm tracking it to update my global install as soon as it's available.

Happy to test edge cases (long context, multi-turn tool calling, vision+text, etc.) if needed. Keep up the amazing work! 🚀

[countzero]

I can reproduce this behaviour with Qwen3.5 architecture models like Qwen3.6-27B-GGUF in OpenCode v1.14.40.

The error message from the llama-server binary is:

Error: Jinja Exception: System message must be at the beginning.

But this is a strict limitation from the official template (https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/7da1103448ba36029c34ce1a9a741dfe93ee0c50/chat_template.jinja#L85) that can be fixed!

Workaround

Clone https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates and use llama-server with the following options:

--chat-template-file 'Qwen-Fixed-Chat-Templates\qwen3.6\chat_template.jinja'

--chat-template-kwargs '{"preserve_thinking":true}'

So maybe not a defect in OpenCode?

This works for me: https://github.com/countzero/windows_llama.cpp/blob/v1.30.0/presets/models_24GB_VRAM.ini#L179-L207

[Milor123]

I can reproduce this behaviour with Qwen3.5 architecture models like Qwen3.6-27B-GGUF in OpenCode v1.14.40.

The error message from the llama-server binary is:

Error: Jinja Exception: System message must be at the beginning.

But this is a strict limitation from the official template (https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/7da1103448ba36029c34ce1a9a741dfe93ee0c50/chat_template.jinja#L85) that can be fixed!

Workaround

Clone https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates and use llama-server with the following options:

--chat-template-file 'Qwen-Fixed-Chat-Templates\qwen3.6\chat_template.jinja'

--chat-template-kwargs '{"preserve_thinking":true}'

So maybe not a defect in OpenCode?

This works for me: https://github.com/countzero/windows_llama.cpp/blob/v1.30.0/presets/models_24GB_VRAM.ini#L179-L207

Did this solution actually work for you? I remember trying to use that same chat template and adding it to my llamaCPP using the --chat-template-file argument, but it didn't work for me; my OpenCode kept giving me the same error.

[countzero]

@Milor123 yes it works:

You can check in the logs if the line srv params_from_: Chat format: peg-native is present. IMHO that indicates that a Jinja template is actually enabled and used.

complete llama-server log output for the above example request

ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24462 MiB):
Device 0: NVIDIA RTX PRO 5000 Blackwell Generation Laptop GPU, compute capability 12.0, VMM: yes, VRAM: 24462 MiB
main: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
build_info: b9053-97f06e9ee
system_info: n_threads = 24 (n_threads_batch = 24) / 24 | CUDA : ARCHS = 1200 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
init: using 23 threads for HTTP server
srv load_models: Loaded 0 cached model presets
srv load_models: Loaded 19 local model presets from D:\AI\LLM\gguf
srv load_models: Loaded 8 custom model presets from D:\Arbeit\windows_llama.cpp\presets\models_24GB_VRAM.ini
srv operator (): Available models (27) (*: custom preset)
srv operator (): * Abiray-Qwen3.6-27B-NVFP4+Qwen3.5-0.8B.IQ4_XS.gguf (aliases: Abiray-Qwen3.6-27B-NVFP4+Qwen3.5-0.8B.IQ4_XS.gguf)
srv operator (): Abiray-Qwen3.6-27B-NVFP4-GGUF
srv operator (): * Abiray-Qwen3.6-27B-NVFP4.gguf (aliases: Abiray-Qwen3.6-27B-NVFP4.gguf)
srv operator (): Bonsai-8B-gguf
srv operator (): Devstral-Small-2-24B-Instruct-2512
srv operator (): GLM-4.7-Flash
srv operator (): LFM2-24B-A2B
srv operator (): Qwen3-Coder-Next
srv operator (): * Qwen3-Coder-Next.IQ4_XS.gguf (aliases: Qwen3-Coder-Next.IQ4_XS.gguf)
srv operator (): Qwen3.5-0.8B
srv operator (): Qwen3.5-122B-A10B
srv operator (): Qwen3.5-27B
srv operator (): Qwen3.5-2B
srv operator (): Qwen3.5-35B-A3B
srv operator (): Qwen3.5-4B
srv operator (): Qwen3.5-9B
srv operator (): Qwen3.6-27B
srv operator (): * Qwen3.6-27B.IQ4_XS+Qwen3.5-0.8B.IQ4_XS.gguf (aliases: Qwen3.6-27B.IQ4_XS+Qwen3.5-0.8B.IQ4_XS.gguf)
srv operator (): * Qwen3.6-27B.IQ4_XS.gguf (aliases: Qwen3.6-27B.IQ4_XS.gguf)
srv operator (): Qwen3.6-35B-A3B
srv operator (): * Qwen3.6-35B-A3B.IQ4_XS.gguf (aliases: Qwen3.6-35B-A3B.IQ4_XS.gguf)
srv operator (): * Qwen3.6-35B-A3B.IQ4_XS.gguf+Qwen3.5-0.8B.IQ4_XS.gguf (aliases: Qwen3.6-35B-A3B.IQ4_XS+Qwen3.5-0.8B.IQ4_XS.gguf)
srv operator (): gemma-4-26B-A4B-it
srv operator (): gemma-4-31B-it
srv operator (): * gemma-4-31B-it.IQ4_XS.gguf (aliases: gemma-4-31B-it.IQ4_XS.gguf)
srv operator (): gpt-oss-120b-GGUF
srv operator (): medgemma-27b-it
main: starting router server, no model will be loaded in this process
start: binding port with default address family
main: router server is listening on http://0.0.0.0:8080
main: NOTE: router mode is experimental
main: it is not recommended to use this mode in untrusted environments
srv ensure_model: model name=Qwen3.6-35B-A3B.IQ4_XS.gguf is not loaded, loading...
srv load: spawning server instance with name=Qwen3.6-35B-A3B.IQ4_XS.gguf on port 54531
srv load: spawning server instance with args:
srv load: D:\Arbeit\windows_llama.cpp\vendor\llama.cpp\build\bin\Release\llama-server.exe
srv load: --chat-template-file
srv load: vendor\Qwen-Fixed-Chat-Templates\qwen3.6\chat_template.jinja
srv load: --chat-template-kwargs
srv load: {"preserve_thinking":true}
srv load: --host
srv load: 127.0.0.1
srv load: --jinja
srv load: --min-p
srv load: 0.0
srv load: --mmproj-offload
srv load: --port
srv load: 54531
srv load: --presence-penalty
srv load: 1.5
srv load: --spec-ngram-mod-n-match
srv load: 24
srv load: --spec-ngram-mod-n-max
srv load: 64
srv load: --spec-ngram-mod-n-min
srv load: 48
srv load: --spec-type
srv load: ngram-mod
srv load: --temperature
srv load: 0.6
srv load: --top-k
srv load: 20
srv load: --top-p
srv load: 0.95
srv load: --alias
srv load: Qwen3.6-35B-A3B.IQ4_XS.gguf
srv load: --ctx-size
srv load: 262144
srv load: --cache-ram
srv load: 51200
srv load: --cache-type-k
srv load: q8_0
srv load: --cache-type-v
srv load: q8_0
srv load: --swa-checkpoints
srv load: 32
srv load: --flash-attn
srv load: true
srv load: --fit-target
srv load: 512
srv load: --kv-unified
srv load: --model
srv load: D:\AI\LLM\gguf\Qwen3.6-35B-A3B\Qwen3.6-35B-A3B.IQ4_XS.gguf
srv load: --mmproj
srv load: D:\AI\LLM\gguf\Qwen3.6-35B-A3B\mmproj.Qwen3.6-35B-A3B.BF16.gguf
srv load: --n-gpu-layers
srv load: -1
srv load: --parallel
srv load: 1
srv load: --reasoning
srv load: on
srv load: --threads
srv load: 24
srv ensure_model: waiting until model name=Qwen3.6-35B-A3B.IQ4_XS.gguf is fully loaded...
srv ensure_model: waiting until model name=Qwen3.6-35B-A3B.IQ4_XS.gguf is fully loaded...
[54531] ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24462 MiB):
[54531] Device 0: NVIDIA RTX PRO 5000 Blackwell Generation Laptop GPU, compute capability 12.0, VMM: yes, VRAM: 24462 MiB
[54531] build_info: b9053-97f06e9ee
[54531] system_info: n_threads = 24 (n_threads_batch = 24) / 24 | CUDA : ARCHS = 1200 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
[54531] init: using 23 threads for HTTP server
[54531] start: binding port with default address family
[54531] main: loading model
[54531] srv load_model: loading model 'D:\AI\LLM\gguf\Qwen3.6-35B-A3B\Qwen3.6-35B-A3B.IQ4_XS.gguf'
[54531] common_init_result: fitting params to device memory, for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on
[54531] common_params_fit_impl: getting device memory data for initial parameters:
[54531] common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
[54531] common_memory_breakdown_print: | - CUDA0 (RTX PRO 5000 Blackwell Generation Laptop GPU) | 24462 = 23087 + (21188 = 17593 + 2782 + 812) + -19812 |
[54531] common_memory_breakdown_print: | - Host | 777 = 257 + 0 + 520 |
[54531] common_params_fit_impl: projected to use 21188 MiB of device memory vs. 23087 MiB of free device memory
[54531] common_params_fit_impl: will leave 1898 >= 512 MiB of free device memory, no changes needed
[54531] common_fit_params: successfully fit params to free device memory
[54531] common_fit_params: fitting params to free memory took 0.87 seconds
[54531] llama_model_loader: loaded meta data with 48 key-value pairs and 733 tensors from D:\AI\LLM\gguf\Qwen3.6-35B-A3B\Qwen3.6-35B-A3B.IQ4_XS.gguf (version GGUF V3 (latest))
[54531] llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
[54531] llama_model_loader: - kv 0: general.architecture str = qwen35moe
[54531] llama_model_loader: - kv 1: general.type str = model
[54531] llama_model_loader: - kv 2: general.sampling.top_k i32 = 20
[54531] llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000
[54531] llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000
[54531] llama_model_loader: - kv 5: general.name str = Qwen3.6 35B A3B
[54531] llama_model_loader: - kv 6: general.basename str = Qwen3.6
[54531] llama_model_loader: - kv 7: general.size_label str = 35B-A3B
[54531] llama_model_loader: - kv 8: general.license str = apache-2.0
[54531] llama_model_loader: - kv 9: general.license.link str = https://huggingface.co/Qwen/Qwen3.6-3...
[54531] llama_model_loader: - kv 10: general.tags arr[str,1] = ["image-text-to-text"]
[54531] llama_model_loader: - kv 11: qwen35moe.block_count u32 = 40
[54531] llama_model_loader: - kv 12: qwen35moe.context_length u32 = 262144
[54531] llama_model_loader: - kv 13: qwen35moe.embedding_length u32 = 2048
[54531] llama_model_loader: - kv 14: qwen35moe.attention.head_count u32 = 16
[54531] llama_model_loader: - kv 15: qwen35moe.attention.head_count_kv u32 = 2
[54531] llama_model_loader: - kv 16: qwen35moe.rope.dimension_sections arr[i32,4] = [11, 11, 10, 0]
[54531] llama_model_loader: - kv 17: qwen35moe.rope.freq_base f32 = 10000000.000000
[54531] llama_model_loader: - kv 18: qwen35moe.attention.layer_norm_rms_epsilon f32 = 0.000001
[54531] llama_model_loader: - kv 19: qwen35moe.expert_count u32 = 256
[54531] llama_model_loader: - kv 20: qwen35moe.expert_used_count u32 = 8
[54531] llama_model_loader: - kv 21: qwen35moe.attention.key_length u32 = 256
[54531] llama_model_loader: - kv 22: qwen35moe.attention.value_length u32 = 256
[54531] llama_model_loader: - kv 23: qwen35moe.expert_feed_forward_length u32 = 512
[54531] llama_model_loader: - kv 24: qwen35moe.expert_shared_feed_forward_length u32 = 512
[54531] llama_model_loader: - kv 25: qwen35moe.ssm.conv_kernel u32 = 4
[54531] llama_model_loader: - kv 26: qwen35moe.ssm.state_size u32 = 128
[54531] llama_model_loader: - kv 27: qwen35moe.ssm.group_count u32 = 16
[54531] llama_model_loader: - kv 28: qwen35moe.ssm.time_step_rank u32 = 32
[54531] llama_model_loader: - kv 29: qwen35moe.ssm.inner_size u32 = 4096
[54531] llama_model_loader: - kv 30: qwen35moe.full_attention_interval u32 = 4
[54531] llama_model_loader: - kv 31: qwen35moe.rope.dimension_count u32 = 64
[54531] llama_model_loader: - kv 32: tokenizer.ggml.model str = gpt2
[54531] llama_model_loader: - kv 33: tokenizer.ggml.pre str = qwen35
[54531] llama_model_loader: - kv 34: tokenizer.ggml.tokens arr[str,248320] = ["!", """, "#", "$", "%", "&", "'", ...
[54531] llama_model_loader: - kv 35: tokenizer.ggml.token_type arr[i32,248320] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
[54531] llama_model_loader: - kv 36: tokenizer.ggml.merges arr[str,247587] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
[54531] llama_model_loader: - kv 37: tokenizer.ggml.eos_token_id u32 = 248046
[54531] llama_model_loader: - kv 38: tokenizer.ggml.padding_token_id u32 = 248044
[54531] llama_model_loader: - kv 39: tokenizer.ggml.bos_token_id u32 = 248044
[54531] llama_model_loader: - kv 40: tokenizer.ggml.add_bos_token bool = false
[54531] llama_model_loader: - kv 41: tokenizer.chat_template str = {%- set image_count = namespace(value...
[54531] llama_model_loader: - kv 42: general.quantization_version u32 = 2
[54531] llama_model_loader: - kv 43: general.file_type u32 = 30
[54531] llama_model_loader: - kv 44: quantize.imatrix.file str = D:\AI\LLM\imatrix\Qwen3.6-35B-A3B.imp...
[54531] llama_model_loader: - kv 45: quantize.imatrix.dataset str = D:\Arbeit\windows_llama.cpp\vendor\wi...
[54531] llama_model_loader: - kv 46: quantize.imatrix.entries_count u32 = 510
[54531] llama_model_loader: - kv 47: quantize.imatrix.chunks_count u32 = 20
[54531] llama_model_loader: - type f32: 301 tensors
[54531] llama_model_loader: - type q5_K: 40 tensors
[54531] llama_model_loader: - type q6_K: 1 tensors
[54531] llama_model_loader: - type iq4_xs: 391 tensors
[54531] print_info: file format = GGUF V3 (latest)
[54531] print_info: file type = IQ4_XS - 4.25 bpw
[54531] print_info: file size = 17.43 GiB (4.32 BPW)
[54531] llama_prepare_model_devices: using device CUDA0 (NVIDIA RTX PRO 5000 Blackwell Generation Laptop GPU) (0000:01:00.0) - 23151 MiB free
[54531] load: 0 unused tokens
[54531] load: printing all EOG tokens:
[54531] load: - 248044 ('<|endoftext|>')
[54531] load: - 248046 ('<|im_end|>')
[54531] load: - 248063 ('<|fim_pad|>')
[54531] load: - 248064 ('<|repo_name|>')
[54531] load: - 248065 ('<|file_sep|>')
[54531] load: special tokens cache size = 33
[54531] load: token to piece cache size = 1.7581 MB
[54531] print_info: arch = qwen35moe
[54531] print_info: vocab_only = 0
[54531] print_info: no_alloc = 0
[54531] print_info: n_ctx_train = 262144
[54531] print_info: n_embd = 2048
[54531] print_info: n_embd_inp = 2048
[54531] print_info: n_layer = 40
[54531] print_info: n_head = 16
[54531] print_info: n_head_kv = 2
[54531] print_info: n_rot = 64
[54531] print_info: n_swa = 0
[54531] print_info: is_swa_any = 0
[54531] print_info: n_embd_head_k = 256
[54531] print_info: n_embd_head_v = 256
[54531] print_info: n_gqa = 8
[54531] print_info: n_embd_k_gqa = 512
[54531] print_info: n_embd_v_gqa = 512
[54531] print_info: f_norm_eps = 0.0e+00
[54531] print_info: f_norm_rms_eps = 1.0e-06
[54531] print_info: f_clamp_kqv = 0.0e+00
[54531] print_info: f_max_alibi_bias = 0.0e+00
[54531] print_info: f_logit_scale = 0.0e+00
[54531] print_info: f_attn_scale = 0.0e+00
[54531] print_info: n_ff = 0
[54531] print_info: n_expert = 256
[54531] print_info: n_expert_used = 8
[54531] print_info: n_expert_groups = 0
[54531] print_info: n_group_used = 0
[54531] print_info: causal attn = 1
[54531] print_info: pooling type = -1
[54531] print_info: rope type = 40
[54531] print_info: rope scaling = linear
[54531] print_info: freq_base_train = 10000000.0
[54531] print_info: freq_scale_train = 1
[54531] print_info: n_ctx_orig_yarn = 262144
[54531] print_info: rope_yarn_log_mul = 0.0000
[54531] print_info: rope_finetuned = unknown
[54531] print_info: mrope sections = [11, 11, 10, 0]
[54531] print_info: ssm_d_conv = 4
[54531] print_info: ssm_d_inner = 4096
[54531] print_info: ssm_d_state = 128
[54531] print_info: ssm_dt_rank = 32
[54531] print_info: ssm_n_group = 16
[54531] print_info: ssm_dt_b_c_rms = 0
[54531] print_info: model type = 35B.A3B
[54531] print_info: model params = 34.66 B
[54531] print_info: general.name = Qwen3.6 35B A3B
[54531] print_info: vocab type = BPE
[54531] print_info: n_vocab = 248320
[54531] print_info: n_merges = 247587
[54531] print_info: BOS token = 248044 '<|endoftext|>'
[54531] print_info: EOS token = 248046 '<|im_end|>'
[54531] print_info: EOT token = 248046 '<|im_end|>'
[54531] print_info: PAD token = 248044 '<|endoftext|>'
[54531] print_info: LF token = 198 'Ċ'
[54531] print_info: FIM PRE token = 248060 '<|fim_prefix|>'
[54531] print_info: FIM SUF token = 248062 '<|fim_suffix|>'
[54531] print_info: FIM MID token = 248061 '<|fim_middle|>'
[54531] print_info: FIM PAD token = 248063 '<|fim_pad|>'
[54531] print_info: FIM REP token = 248064 '<|repo_name|>'
[54531] print_info: FIM SEP token = 248065 '<|file_sep|>'
[54531] print_info: EOG token = 248044 '<|endoftext|>'
[54531] print_info: EOG token = 248046 '<|im_end|>'
[54531] print_info: EOG token = 248063 '<|fim_pad|>'
[54531] print_info: EOG token = 248064 '<|repo_name|>'
[54531] print_info: EOG token = 248065 '<|file_sep|>'
[54531] print_info: max token length = 256
[54531] load_tensors: loading model tensors, this can take a while... (mmap = true, direct_io = false)
[54531] load_tensors: offloading output layer to GPU
[54531] load_tensors: offloading 39 repeating layers to GPU
[54531] load_tensors: offloaded 41/41 layers to GPU
[54531] load_tensors: CPU_Mapped model buffer size = 257.66 MiB
[54531] load_tensors: CUDA0 model buffer size = 17593.02 MiB
[54531] ...................................................................................................
[54531] common_init_result: added <|endoftext|> logit bias = -inf
[54531] common_init_result: added <|im_end|> logit bias = -inf
[54531] common_init_result: added <|fim_pad|> logit bias = -inf
[54531] common_init_result: added <|repo_name|> logit bias = -inf
[54531] common_init_result: added <|file_sep|> logit bias = -inf
[54531] llama_context: constructing llama_context
[54531] llama_context: n_seq_max = 1
[54531] llama_context: n_ctx = 262144
[54531] llama_context: n_ctx_seq = 262144
[54531] llama_context: n_batch = 2048
[54531] llama_context: n_ubatch = 512
[54531] llama_context: causal_attn = 1
[54531] llama_context: flash_attn = enabled
[54531] llama_context: kv_unified = true
[54531] llama_context: freq_base = 10000000.0
[54531] llama_context: freq_scale = 1
[54531] llama_context: CUDA_Host output buffer size = 0.95 MiB
[54531] llama_kv_cache: CUDA0 KV buffer size = 2720.00 MiB
[54531] llama_kv_cache: size = 2720.00 MiB (262144 cells, 10 layers, 1/1 seqs), K (q8_0): 1360.00 MiB, V (q8_0): 1360.00 MiB
[54531] llama_kv_cache: attn_rot_k = 1, n_embd_head_k_all = 256
[54531] llama_kv_cache: attn_rot_v = 1, n_embd_head_k_all = 256
[54531] llama_memory_recurrent: CUDA0 RS buffer size = 62.81 MiB
[54531] llama_memory_recurrent: size = 62.81 MiB ( 1 cells, 40 layers, 1 seqs), R (f32): 2.81 MiB, S (f32): 60.00 MiB
[54531] sched_reserve: reserving ...
[54531] sched_reserve: resolving fused Gated Delta Net support:
[54531] sched_reserve: fused Gated Delta Net (autoregressive) enabled
[54531] sched_reserve: fused Gated Delta Net (chunked) enabled
[54531] sched_reserve: CUDA0 compute buffer size = 812.28 MiB
[54531] sched_reserve: CUDA_Host compute buffer size = 520.29 MiB
[54531] sched_reserve: graph nodes = 3849
[54531] sched_reserve: graph splits = 2
[54531] sched_reserve: reserve took 68.80 ms, sched copies = 1
[54531] common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
[54531] clip_model_loader: model name: Qwen3.6 35B A3B
[54531] clip_model_loader: description:
[54531] clip_model_loader: GGUF version: 3
[54531] clip_model_loader: alignment: 32
[54531] clip_model_loader: n_tensors: 334
[54531] clip_model_loader: n_kv: 29
[54531]
[54531] clip_model_loader: has vision encoder
[54531] clip_ctx: CLIP using CUDA0 backend
[54531] load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
[54531] load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
[54531] load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842
[54531]
[54531] load_hparams: projector: qwen3vl_merger
[54531] load_hparams: n_embd: 1152
[54531] load_hparams: n_head: 16
[54531] load_hparams: n_ff: 4304
[54531] load_hparams: n_layer: 27
[54531] load_hparams: ffn_op: gelu
[54531] load_hparams: projection_dim: 2048
[54531]
[54531] --- vision hparams ---
[54531] load_hparams: image_size: 768
[54531] load_hparams: patch_size: 16
[54531] load_hparams: has_llava_proj: 0
[54531] load_hparams: minicpmv_version: 0
[54531] load_hparams: n_merge: 2
[54531] load_hparams: n_wa_pattern: 0
[54531] load_hparams: image_min_pixels: 8192
[54531] load_hparams: image_max_pixels: 4194304
[54531]
[54531] load_hparams: model size: 860.98 MiB
[54531] load_hparams: metadata size: 0.12 MiB
[54531] warmup: warmup with image size = 1472 x 1472
[54531] alloc_compute_meta: CUDA0 compute buffer size = 248.10 MiB
[54531] alloc_compute_meta: CPU compute buffer size = 24.93 MiB
[54531] alloc_compute_meta: graph splits = 1, nodes = 823
[54531] warmup: flash attention is enabled
[54531] srv load_model: loaded multimodal model, 'D:\AI\LLM\gguf\Qwen3.6-35B-A3B\mmproj.Qwen3.6-35B-A3B.BF16.gguf'
[54531] srv load_model: initializing slots, n_slots = 1
[54531] common_context_can_seq_rm: the target context does not support partial sequence removal
[54531] srv load_model: speculative decoding will use checkpoints
[54531] common_speculative_init: initialized ngram_mod with n_match=24, size=4194304 (16.000 MB)
[54531] slot load_model: id 0 | task -1 | speculative decoding context initialized
[54531] slot load_model: id 0 | task -1 | new slot, n_ctx = 262144
[54531] srv load_model: prompt cache is enabled, size limit: 51200 MiB
[54531] srv load_model: use --cache-ram 0 to disable the prompt cache
[54531] srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
[54531] srv init: init: idle slots will be saved to prompt cache and cleared upon starting a new task
[54531] init: chat template, example_format: '<|im_start|>system
[54531] You are a helpful assistant<|im_end|>
[54531] <|im_start|>user
[54531] Hello<|im_end|>
[54531] <|im_start|>assistant
[54531] Hi there<|im_end|>
[54531] <|im_start|>user
[54531] How are you?<|im_end|>
[54531] <|im_start|>assistant
[54531]
[54531] '
[54531] srv init: init: chat template, thinking = 1
[54531] main: model loaded
[54531] main: server is listening on http://127.0.0.1:54531
[54531] main: starting the main loop...
[54531] cmd_child_to_router:ready
srv proxy_reques: proxying request to model Qwen3.6-35B-A3B.IQ4_XS.gguf on port 54531
srv proxy_reques: proxying request to model Qwen3.6-35B-A3B.IQ4_XS.gguf on port 54531
[54531] srv operator (): child server monitoring thread started, waiting for EOF on stdin...
[54531] srv update_slots: all slots are idle
[54531] srv params_from_: Chat format: peg-native
[54531] slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
[54531] srv get_availabl: updating prompt cache
[54531] srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
[54531] srv update: - cache state: 0 prompts, 0.000 MiB (limits: 51200.000 MiB, 262144 tokens, 53687091200 est)
[54531] srv get_availabl: prompt cache update took 0.01 ms
[54531] slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> temp-ext -> dist
[54531] slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
[54531] slot update_slots: id 0 | task 0 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 865
[54531] slot update_slots: id 0 | task 0 | n_tokens = 0, memory_seq_rm [0, end)
[54531] slot update_slots: id 0 | task 0 | prompt processing progress, n_tokens = 349, batch.n_tokens = 349, progress = 0.403468
[54531] srv params_from_: Chat format: peg-native
[54531] slot update_slots: id 0 | task 0 | n_tokens = 349, memory_seq_rm [349, end)
[54531] slot update_slots: id 0 | task 0 | prompt processing progress, n_tokens = 861, batch.n_tokens = 512, progress = 0.995376
[54531] slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 348, pos_max = 348, n_tokens = 349, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 0 | n_tokens = 861, memory_seq_rm [861, end)
[54531] slot init_sampler: id 0 | task 0 | init sampler, took 0.09 ms, tokens: text = 865, total = 865
[54531] slot update_slots: id 0 | task 0 | prompt processing done, n_tokens = 865, batch.n_tokens = 4
[54531] slot create_check: id 0 | task 0 | created context checkpoint 2 of 32 (pos_min = 860, pos_max = 860, n_tokens = 861, size = 62.813 MiB)
[54531] begin: ngram_mod occupancy = 841/4194304 (0.00)
[54531] srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
[54531] slot print_timing: id 0 | task 0 |
[54531] prompt eval time = 408.23 ms / 865 tokens ( 0.47 ms per token, 2118.92 tokens per second)
[54531] eval time = 2170.49 ms / 286 tokens ( 7.59 ms per token, 131.77 tokens per second)
[54531] total time = 2578.72 ms / 1151 tokens
[54531] statistics ngram_mod: #calls(b,g,a) = 1 285 0, #gen drafts = 0, #acc drafts = 0, #gen tokens = 0, #acc tokens = 0, dur(b,g,a) = 0.102, 0.267, 0.000 ms
[54531] slot release: id 0 | task 0 | stop processing: n_tokens = 1150, truncated = 0
[54531] slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 19491464
[54531] srv get_availabl: updating prompt cache
[54531] srv prompt_save: - saving prompt with length 1150, total state size = 74.768 MiB
[54531] srv load: - looking for better prompt, base f_keep = 0.003, sim = 0.000
[54531] srv update: - cache state: 1 prompts, 200.394 MiB (limits: 51200.000 MiB, 262144 tokens, 293820 est)
[54531] srv update: - prompt 000001C5B491D500: 1150 tokens, checkpoints: 2, 200.394 MiB
[54531] srv get_availabl: prompt cache update took 57.04 ms
[54531] reasoning-budget: activated, budget=2147483647 tokens
[54531] slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> temp-ext -> dist
[54531] slot launch_slot_: id 0 | task 2 | processing task, is_child = 0
[54531] slot update_slots: id 0 | task 2 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 30189
[54531] slot update_slots: id 0 | task 2 | n_past = 3, slot.prompt.tokens.size() = 1150, seq_id = 0, pos_min = 1149, n_swa = 0
[54531] slot update_slots: id 0 | task 2 | Checking checkpoint with [860, 860] against 3...
[54531] slot update_slots: id 0 | task 2 | Checking checkpoint with [348, 348] against 3...
[54531] slot update_slots: id 0 | task 2 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
[54531] slot update_slots: id 0 | task 2 | erased invalidated context checkpoint (pos_min = 348, pos_max = 348, n_tokens = 349, n_swa = 0, pos_next = 0, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | erased invalidated context checkpoint (pos_min = 860, pos_max = 860, n_tokens = 861, n_swa = 0, pos_next = 0, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | n_tokens = 0, memory_seq_rm [0, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.067839
[54531] slot update_slots: id 0 | task 2 | n_tokens = 2048, memory_seq_rm [2048, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 4096, batch.n_tokens = 2048, progress = 0.135679
[54531] slot update_slots: id 0 | task 2 | n_tokens = 4096, memory_seq_rm [4096, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 6144, batch.n_tokens = 2048, progress = 0.203518
[54531] slot update_slots: id 0 | task 2 | n_tokens = 6144, memory_seq_rm [6144, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 8192, batch.n_tokens = 2048, progress = 0.271357
[54531] slot update_slots: id 0 | task 2 | n_tokens = 8192, memory_seq_rm [8192, end)
[54531] slot update_slots: id 0 | task 2 | 8192 tokens since last checkpoint at 0, creating new checkpoint during processing at position 10240
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 10240, batch.n_tokens = 2048, progress = 0.339196
[54531] slot create_check: id 0 | task 2 | created context checkpoint 1 of 32 (pos_min = 8191, pos_max = 8191, n_tokens = 8192, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | n_tokens = 10240, memory_seq_rm [10240, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 12288, batch.n_tokens = 2048, progress = 0.407036
[54531] slot update_slots: id 0 | task 2 | n_tokens = 12288, memory_seq_rm [12288, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 14336, batch.n_tokens = 2048, progress = 0.474875
[54531] slot update_slots: id 0 | task 2 | n_tokens = 14336, memory_seq_rm [14336, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 16384, batch.n_tokens = 2048, progress = 0.542714
[54531] slot update_slots: id 0 | task 2 | n_tokens = 16384, memory_seq_rm [16384, end)
[54531] slot update_slots: id 0 | task 2 | 8192 tokens since last checkpoint at 8192, creating new checkpoint during processing at position 18432
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 18432, batch.n_tokens = 2048, progress = 0.610554
[54531] slot create_check: id 0 | task 2 | created context checkpoint 2 of 32 (pos_min = 16383, pos_max = 16383, n_tokens = 16384, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | n_tokens = 18432, memory_seq_rm [18432, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 20480, batch.n_tokens = 2048, progress = 0.678393
[54531] slot update_slots: id 0 | task 2 | n_tokens = 20480, memory_seq_rm [20480, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 22528, batch.n_tokens = 2048, progress = 0.746232
[54531] slot update_slots: id 0 | task 2 | n_tokens = 22528, memory_seq_rm [22528, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 24576, batch.n_tokens = 2048, progress = 0.814071
[54531] slot update_slots: id 0 | task 2 | n_tokens = 24576, memory_seq_rm [24576, end)
[54531] slot update_slots: id 0 | task 2 | 8192 tokens since last checkpoint at 16384, creating new checkpoint during processing at position 26624
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 26624, batch.n_tokens = 2048, progress = 0.881911
[54531] slot create_check: id 0 | task 2 | created context checkpoint 3 of 32 (pos_min = 24575, pos_max = 24575, n_tokens = 24576, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | n_tokens = 26624, memory_seq_rm [26624, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 28672, batch.n_tokens = 2048, progress = 0.949750
[54531] slot update_slots: id 0 | task 2 | n_tokens = 28672, memory_seq_rm [28672, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 29673, batch.n_tokens = 1001, progress = 0.982908
[54531] slot update_slots: id 0 | task 2 | n_tokens = 29673, memory_seq_rm [29673, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 30185, batch.n_tokens = 512, progress = 0.999867
[54531] slot create_check: id 0 | task 2 | created context checkpoint 4 of 32 (pos_min = 29672, pos_max = 29672, n_tokens = 29673, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | n_tokens = 30185, memory_seq_rm [30185, end)
[54531] slot init_sampler: id 0 | task 2 | init sampler, took 2.85 ms, tokens: text = 30189, total = 30189
[54531] slot update_slots: id 0 | task 2 | prompt processing done, n_tokens = 30189, batch.n_tokens = 4
[54531] slot create_check: id 0 | task 2 | created context checkpoint 5 of 32 (pos_min = 30184, pos_max = 30184, n_tokens = 30185, size = 62.813 MiB)
[54531] begin: ngram_mod occupancy = 24353/4194304 (0.01)
[54531] ~llama_io_write_device: allocated 'CUDA0' buffer 62.812 MiB
[54531] srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
[54531] reasoning-budget: deactivated (natural end)
[54531] slot print_timing: id 0 | task 2 |
[54531] prompt eval time = 11103.66 ms / 30189 tokens ( 0.37 ms per token, 2718.83 tokens per second)
[54531] eval time = 895.26 ms / 84 tokens ( 10.66 ms per token, 93.83 tokens per second)
[54531] total time = 11998.92 ms / 30273 tokens
[54531] draft acceptance rate = 0.03125 ( 2 accepted / 64 generated)
[54531] statistics ngram_mod: #calls(b,g,a) = 2 366 1, #gen drafts = 1, #acc drafts = 1, #gen tokens = 64, #acc tokens = 2, dur(b,g,a) = 2.067, 0.353, 0.000 ms
[54531] slot release: id 0 | task 2 | stop processing: n_tokens = 30272, truncated = 0
[54531] srv update_slots: all slots are idle
srv proxy_reques: proxying request to model Qwen3.6-35B-A3B.IQ4_XS.gguf on port 54531
[54531] srv params_from_: Chat format: peg-native
[54531] slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.996 (> 0.100 thold), f_keep = 1.000
[54531] reasoning-budget: activated, budget=2147483647 tokens
[54531] slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> temp-ext -> dist
[54531] slot launch_slot_: id 0 | task 389 | processing task, is_child = 0
[54531] slot update_slots: id 0 | task 389 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 30404
[54531] slot update_slots: id 0 | task 389 | n_tokens = 30272, memory_seq_rm [30272, end)
[54531] slot update_slots: id 0 | task 389 | prompt processing progress, n_tokens = 30400, batch.n_tokens = 128, progress = 0.999868
[54531] slot create_check: id 0 | task 389 | created context checkpoint 6 of 32 (pos_min = 30271, pos_max = 30271, n_tokens = 30272, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 389 | n_tokens = 30400, memory_seq_rm [30400, end)
[54531] slot init_sampler: id 0 | task 389 | init sampler, took 2.92 ms, tokens: text = 30404, total = 30404
[54531] slot update_slots: id 0 | task 389 | prompt processing done, n_tokens = 30404, batch.n_tokens = 4
[54531] slot create_check: id 0 | task 389 | created context checkpoint 7 of 32 (pos_min = 30399, pos_max = 30399, n_tokens = 30400, size = 62.813 MiB)
[54531] begin: ngram_mod occupancy = 24565/4194304 (0.01)
[54531] srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
[54531] reasoning-budget: deactivated (natural end)
[54531] slot print_timing: id 0 | task 389 |
[54531] prompt eval time = 162.66 ms / 132 tokens ( 1.23 ms per token, 811.53 tokens per second)
[54531] eval time = 575.59 ms / 47 tokens ( 12.25 ms per token, 81.66 tokens per second)
[54531] total time = 738.25 ms / 179 tokens
[54531] draft acceptance rate = 0.03125 ( 2 accepted / 64 generated)
[54531] statistics ngram_mod: #calls(b,g,a) = 3 410 2, #gen drafts = 2, #acc drafts = 2, #gen tokens = 128, #acc tokens = 4, dur(b,g,a) = 4.041, 0.413, 0.000 ms
[54531] slot release: id 0 | task 389 | stop processing: n_tokens = 30450, truncated = 0
[54531] srv update_slots: all slots are idle
srv proxy_reques: proxying request to model Qwen3.6-35B-A3B.IQ4_XS.gguf on port 54531
[54531] srv params_from_: Chat format: peg-native
[54531] slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.885 (> 0.100 thold), f_keep = 1.000
[54531] reasoning-budget: activated, budget=2147483647 tokens
[54531] slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> temp-ext -> dist
[54531] slot launch_slot_: id 0 | task 437 | processing task, is_child = 0
[54531] slot update_slots: id 0 | task 437 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 34412
[54531] slot update_slots: id 0 | task 437 | n_tokens = 30450, memory_seq_rm [30450, end)
[54531] slot update_slots: id 0 | task 437 | prompt processing progress, n_tokens = 32498, batch.n_tokens = 2048, progress = 0.944380
[54531] slot update_slots: id 0 | task 437 | n_tokens = 32498, memory_seq_rm [32498, end)
[54531] slot update_slots: id 0 | task 437 | prompt processing progress, n_tokens = 33896, batch.n_tokens = 1398, progress = 0.985005
[54531] slot update_slots: id 0 | task 437 | n_tokens = 33896, memory_seq_rm [33896, end)
[54531] slot update_slots: id 0 | task 437 | prompt processing progress, n_tokens = 34408, batch.n_tokens = 512, progress = 0.999884
[54531] slot create_check: id 0 | task 437 | created context checkpoint 8 of 32 (pos_min = 33895, pos_max = 33895, n_tokens = 33896, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 437 | n_tokens = 34408, memory_seq_rm [34408, end)
[54531] slot init_sampler: id 0 | task 437 | init sampler, took 3.25 ms, tokens: text = 34412, total = 34412
[54531] slot update_slots: id 0 | task 437 | prompt processing done, n_tokens = 34412, batch.n_tokens = 4
[54531] slot create_check: id 0 | task 437 | created context checkpoint 9 of 32 (pos_min = 34407, pos_max = 34407, n_tokens = 34408, size = 62.813 MiB)
[54531] begin: ngram_mod occupancy = 28493/4194304 (0.01)
[54531] srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
[54531] reasoning-budget: deactivated (natural end)
[54531] slot print_timing: id 0 | task 437 |
[54531] prompt eval time = 1684.01 ms / 3962 tokens ( 0.43 ms per token, 2352.71 tokens per second)
[54531] eval time = 2114.12 ms / 209 tokens ( 10.12 ms per token, 98.86 tokens per second)
[54531] total time = 3798.13 ms / 4171 tokens
[54531] statistics ngram_mod: #calls(b,g,a) = 4 618 2, #gen drafts = 2, #acc drafts = 2, #gen tokens = 128, #acc tokens = 4, dur(b,g,a) = 6.291, 0.634, 0.000 ms
[54531] slot release: id 0 | task 437 | stop processing: n_tokens = 34620, truncated = 0
[54531] srv update_slots: all slots are idle