reasoning_budget_tokens is ignored?

Heyo. Just downloaded, so v0.1.39. Ran the setup.sh on uBuntu 26, all good.

[strata] logging output shows 'thinking: x of max 6400 tokens'. I set "reasoning_budget_tokens": 2048 in the config, but it still thinks up to 6400 tokens, rather than 2048 as set in the config:

{
 "exe": "...",
 "args": [
  "--pack",
  "~/devl/AI/Strata-data/packs/iq2_xs",
  "--native",
  "~/devl/AI/Strata-data/models/IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf",
  "--ple-gguf",
  "~/devl/AI/Strata-data/models/IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf",
  "--expert-profile",
  "~/devl/AI/Strata/data/expert-profile.bin",
  "--expert-cache",
  "auto",
  "--prefill",
  "auto",
  "--spec",
  "4",
  "--mtp",
  "~/devl/AI/Strata-data/mtp/rt",
  "--max-context",
  "65535",
  "--kv",
  "int8",
  "--spec-min-p",
  "0.5",
  "--pool-workers",
  "10"
],
 "model_name": "qwen3.8-flash-next-iq2_xs",
 "port": 11433,
 "gpu": 0,
 "gpus_asked": true,
 "reasoning_budget_tokens": 2048
}

Interestingly, the logging also shows answering: x of max 6400 tokens, so im not sure where the 6400 is coming from?

Right now, the model just fills itself up thinking then fails to answer.

Some data from another 0.1.39 install, and then my opinion on what is happening in your case.

What I checked (0.1.39, Linux)

The budget logic passes on 0.1.39. python -m unittest serve.test_server.ThinkingBudget: 9/9 OK. That includes test_the_config_default_and_a_request_that_turns_it_off, which covers the config's reasoning_budget_tokens.
It cuts live. I sent a request with "reasoning_budget_tokens": 64 and max_tokens: 3000 to a running server. The log showed:
```
[strata] thinking budget reached (65 tokens): wrapping up the thinking
[strata] done: 408 tokens in 6 s (70.4 tok/s) (stop, cancel=False)
```
The model then answered normally (finish_reason: stop).
The config budget loads at start. We run with "reasoning_budget_tokens": 16000 in strata-*.json. The start log prints:
```
[strata] thinking budget: 16000 tokens (reasoning_budget_tokens; a request can set its own)
```
On 0.1.38 the same config cut real agent replies twice (thinking budget reached (16000 tokens)).
"of max 6400" is not the budget. That progress line prints the request's max_tokens ({generated} of max {max_tokens} in serve/server.py). It comes from your client or the chat's "max" setting.

My opinion (not verified on your machine)

Your symptom, thinking until the limit with no answer, looks like the budget is not active for those requests. The log then says "the reply reached max tokens while still thinking, so it has no answer". The mechanism itself works, so I would check, in this order:

Restart the server after editing the config. The budget is read only at start. Look for the line [strata] thinking budget: 2048 tokens … in the window. If it is not there, the budget is not loaded.
Make sure the file you edited is the one in use. The start script passes --config strata-<model>.json.
Check whether your client sends its own reasoning_budget_tokens. A request's value overrides the config, and 0 turns the budget off.
On the value itself: 2048 is quite short for this model's thinking. In my 64-token test above, the cut-short reasoning produced a wrong answer: it wrote 466 − 66 = 171, while the correct result is 400. For hard questions I would start higher (4–8K for chat; we use 16K for an agent) and keep max_tokens well above the budget, so there is room left to answer.

One more idea to isolate it quickly: try the same prompt in Strata's own web chat (open http://127.0.0.1:11433/ in a browser; your port is 11433).

The web chat never sends reasoning_budget_tokens; it sends only reasoning_effort, the temperature and max_tokens (see serve/web/app.js). So there the config budget is the only one that applies. In the chat settings:

set the thinking level to anything except off (with thinking off there is nothing to limit);
set "max" well above 2048.

Then ask something that makes it think and watch the server window:

[strata] thinking budget reached (2048 tokens): wrapping up the thinking appears: the server and the config are fine. The problem is in your agent/client, which most likely sends its own reasoning_budget_tokens (or 0, which turns the budget off).
The line does not appear, and the start log has no [strata] thinking budget: 2048 tokens line: the config is not loaded. Check the restart and which --config the start script uses.
The start line is there but the chat still never cuts: that would be a real bug, and worth posting with that log here.

The same check without a browser:

curl -s http://127.0.0.1:11433/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model":"m","max_tokens":6000,"messages":[{"role":"user","content":"How many positive integers below 1000 are divisible by 3 or 5 but not by 7? Think carefully."}]}' > /dev/null

(add -H "Authorization: Bearer <key>" if you set an API key), then look for the same log line.

The 6400 in the log line is the request's max_tokens, not the budget. Check the start log for '[strata] thinking budget: 2048 tokens (reasoning_budget_tokens ...)' after a restart: the config is read only at start. A request's own reasoning_budget_tokens overrides the config, and 0 turns it off, so a client that sends one would explain this. If the start line is there and a request without its own value never cuts at 2048, please post the log: that would be a real bug.

Ahh ok, so the client can override the reasoning_budget from the config. That explains some things :) I would have thought the config would have overridden, so i guess that's good, means it's more configurable on the fly.

I used the curl above and indeed it showed max_tokens in the logs. And indeed on start it does display the configured reasoning_budget.

Thanks for your help guys, i'll run with this now and see how I go.

cheers

reasoning_budget_tokens is a different setting from the 6400 in your log, which is max_tokens. If the budget is not the value you set, please send the first lines of the server log.

I think I'm seeing this well.
I'm also seeing as 3 different numbers.

json config

{
 "exe": "C:\\Strata\\Strata\\engine\\strata.exe",
 "args": [
  "--pack",
  "C:\\Strata\\Strata-data\\packs\\unsloth-ud-iq4_xs",
  "--native",
  "C:\\Strata\\Strata-data\\models\\unsloth-UD-IQ4_XS\\Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf",
  "--expert-profile",
  "C:\\Strata\\Strata\\data\\expert-profile.bin",
  "--expert-cache",
  "auto",
  "--prefill",
  "auto",
  "--spec",
  "4",
  "--mtp",
  "C:\\Strata\\Strata-data\\mtp\\rt",
  "--max-context",
  "262144",
  "--kv",
  "int8",
  "--resident-budget-gib",
  "55",
  "--spec-min-p",
  "0.5",
  "--adapt-every",
  "1",
  "--adapt-swaps",
  "160",
  "--adapt-decay",
  "0.97"
 ],
 "cwd": "C:\\Strata\\Strata",
 "tokenizer": "C:\\Strata\\Strata-data\\packs\\unsloth-ud-iq4_xs\\tokenizer",
 "model_name": "qwen3.8-flash-next-unsloth-ud-iq4_xs",
 "log": "C:\\Strata\\Strata\\strata-unsloth-ud-iq4_xs.log",
 "lib_dirs": [
  "C:\\Strata\\Strata\\.venv\\Lib\\site-packages\\nvidia\\cu13\\bin\\x86_64"
 ],
 "port": 8080,
 "gpu": 0,
 "gpus_asked": true,
 "host": "0.0.0.0",
 "api_key": "gibble",
 "fit_max_tokens": true,
 "reasoning_budget_tokens": 131072
}

first lines of start up

  ----------------------------------------------------------------------------------------------------
  Starting qwen3.8-flash-next-unsloth-ud-iq4_xs: it loads 34-55 GB into RAM and locks part of it for the GPU.
  While it does, YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES (longer the first time after a
  restart). That is normal: please wait and don't close this window - the browser opens when it is ready.
  Later, closing this window stops the model.
  ----------------------------------------------------------------------------------------------------
  Settings (strata-unsloth-ud-iq4_xs.json): --expert-cache auto --prefill auto --spec 4 --max-context
    262144 --kv int8 --resident-budget-gib 55 --spec-min-p 0.5 --adapt-every 1 --adapt-swaps 160
    --adapt-decay 0.97; server 0.0.0.0:8080, api key set, gpu 0, fit_max_tokens true,
    reasoning_budget_tokens 32768
loading the model (the first start takes a minute or two) ...

error/suggestion after a truncated response.

[strata] done: 4096 tokens in 66 s (62.8 tok/s) (length, cancel=False), expert cache 68.9% hit (+22.0% of the routed experts over PCIe)
[strata] the reply reached max tokens while still thinking, so it has no answer: a thinking budget (reasoning_budget_tokens, in the request or in strata-<model>.json for every request) leaves room to answer
[strata] thinking: 26 of max 4096 tokens, 43.7 tok/s,

Progress line shows the thinking budget, not max_tokens, while one ap…

Cap a thinking budget at max_tokens less an answer reserve, so the re…

@Yassim

Thanks for the logs. The of max 6400 on the thinking: line is the request's max_tokens, not the budget. We changed that line: while a thinking budget applies it now reads thinking: 120 of budget 2048 (max 6400) tokens, and answering: goes back to of max N after the wrap-up. The config's reasoning_budget_tokens applies to every request that does not send its own value (a request's own value wins, and 0 turns it off). In v0.1.42.

@Yassim: Your three numbers: 131072 is the budget in your json, 4096 is the request's max_tokens (the client sets it), and 32768 in the Settings line. We found no clamp anywhere: that line prints the value of the config file as read at that start, so the file held 32768 then (or the window is from before your edit; the config is read only at start, so restart after editing). The real problem is the one you hit: a budget (131072) above max_tokens (4096) can never fire, so the reply ended while still thinking. Fixed in v0.1.42: when a budget is at or above what max_tokens leaves room for, the thinking is now closed early enough to leave room to answer (max_tokens minus a reserve of max(512, a quarter of max_tokens)), and the log says thinking capped at 3072 of max_tokens 4096 to leave room for the answer (budget 131072). A budget well below max_tokens behaves as before. For long thinking, raise max_tokens in the client too.

@Niko1221

@Niko1221 Yep, Ive just found the param to fix. It was in my Pi agents config. It was seniding a max_tokens of the 4k, which ment it kept running out of room when doing some refactoring.

BTW, Thanks for this project!

進み具合の行に、考えの予算があるときは max_tokens でなくその予算を出す（Niko1221#984）（upstream bce19db

考えの予算を、答えの分を残して max_tokens 以下に抑え、考えている途中で返事が切れないようにする（Niko1221#984）（u…

Merge upstream v0.1.43 into windows-coding

3. The 6400 in the log line is the request's max_tokens, not the budget. Check the start log for '[strata] thinking budget: 2048 tokens (reasoning_budget_tokens ...)' after a restart: the config is read only at start. A request's own reasoning_budget_tokens overrides the config, and 0 turns it off, so a client that sends one would explain this. If the start line is there and a request without its own value never cuts at 2048, please post the log: that would be a real bug.

pi coding agent just changed their json schema for models.json and limit the thinkingTokenBudgetField to one of thinking_token_budget, thinking_budget or thinking_budget_tokens, maybe we should change field name to follow it.

        "thinkingTokenBudgetField": {
          "anyOf": [
            {
              "type": "string",
              "const": "thinking_token_budget"
            },
            {
              "type": "string",
              "const": "thinking_budget"
            },
            {
              "type": "string",
              "const": "thinking_budget_tokens"
            }
          ],
          "description": "Top-level request field used to cap reasoning tokens from thinkingBudgets. Reasoning and the answer share max_tokens on these endpoints. \"thinking_token_budget\" is vLLM, \"thinking_budget\" is Qwen, DashScope, or SGLang, and \"thinking_budget_tokens\" is llama.cpp. Off by default and not set on the generated catalog."
        }

Thanks for the report. To keep the tracker current we are closing issues older than three days: 0.1.44 is out with a long list of fixes, and many older reports are covered by it or by 0.1.43.1.

If this still happens on 0.1.44, please leave a comment here with your engine log (and the version line it prints), and we will reopen it. A new issue that links this one works too.

Progress line shows the thinking budget, not max_tokens, while one ap…

Cap a thinking budget at max_tokens less an answer reserve, so the re…

feat(serve): upstream v0.1.45 server, tokenizer and observability enh…

Navigation Menu

Repository navigation

reasoning_budget_tokens is ignored? #984

What I checked (0.1.39, Linux)

My opinion (not verified on your machine)

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

reasoning_budget_tokens is ignored? #984

Description

Activity

talisp commented on Oct 5, 2026

What I checked (0.1.39, Linux)

My opinion (not verified on your machine)

talisp commented on Oct 5, 2026

Niko1221 commented on Oct 6, 2026

PentaForgeDev commented on Oct 7, 2026

Niko1221 commented on Oct 7, 2026

Yassim commented on Oct 9, 2026

Niko1221 commented on Oct 10, 2026

Yassim commented on Oct 11, 2026

shizhx commented on Oct 13, 2026

Niko1221 commented on Oct 13, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions