Repository navigation
Feature request: cap on thinking tokens (like llama.cpp's reasoning_budget_tokens) #123
Description
With reasoning_effort medium, Qwen3.8-Flash-Next sometimes thinks past 8k to 12k tokens. Thinking and the answer share max_tokens, so some requests end with reasoning only and empty content, and the client sees no answer.
llama.cpp handles this with reasoning_budget_tokens and reasoning_budget_message: once the budget is spent, the model is told to wrap up, thinking is closed, and it writes the answer. Could Strata offer something similar, either per request or as a server default?
Setup: Linux, RTX 4080 Laptop 12 GB, IQ3_S, engine 0.1.20.
Activity
technerd-official commentedon Sep 29, 2026
Just found out that you can set "Max tokens" inside the chat window "127.0.0.1:8080" (click on the small gear) right now max tokens is set to "Until done" and also the sampling settings can be adjusted. These carry over to an agent in case you use one.
Niko1221 commentedon Sep 29, 2026
Good idea - planned server-side: when the thinking part reaches the budget, append a short wrap-up and
</think>, and continue from there (the prefix is reused, so it costs nothing extra). Keeping this open until it lands.Niko1221 commentedon Sep 29, 2026
Planned as described: once the thinking reaches the budget, the server appends a short wrap-up and
</think>and continues from the reused prefix. Until then, the chat page's max-tokens setting (the gear icon) helps.Niko1221 commentedon Sep 30, 2026
Planned for 0.1.31 as an opt-in: a
reasoning_budget_tokensrequest field (and a config default). At the budget the server closes the thinking block and lets the model answer. Off by default.serve: an opt-in hard thinking budget, reasoning_budget_tokens (#123)
Niko1221 commentedon Oct 1, 2026
0.1.31:
reasoning_budget_tokensin a request (OpenAI or Anthropic) or as a config default, off by default. At the budget the server closes the thinking block with a short wrap-up and lets the model answer. See docs/DETAILS.md.Strata-Qwen36: reasoning levels as thinking budgets; chat.py on port …
#278 follow-up: an Anthropic request with its own reasoning_budget_to…
serve: an opt-in hard thinking budget, reasoning_budget_tokens (#123)
#278 follow-up: an Anthropic request with its own reasoning_budget_to…
serve: an opt-in hard thinking budget, reasoning_budget_tokens (Niko1…
Niko1221#278 follow-up: an Anthropic request with its own reasoning_b…
serve: an opt-in hard thinking budget, reasoning_budget_tokens (Niko1…
Niko1221#278 follow-up: an Anthropic request with its own reasoning_b…
serve: reasoning_budget_share caps the thinking at a share of max_tok…
serve: a reply that ends inside its thinking with no answer is contin…
serve: a reply that ends inside its thinking with no answer is contin…
serve: a reply that ends inside its thinking with no answer is contin…
serve: a reply that ends inside its thinking with no answer is contin…
serve: a reply that ends inside its thinking with no answer is contin…
serve: eos-lenient — a stop token the model wrote ABOUT names itself …
serve: a reply that ends inside its thinking with no answer is contin…
serve: a reply that ends inside its thinking with no answer is contin…
serve: DeepSeek-V4.1 with a short max_tokens (48-1023) caps the think…
serve: DeepSeek-V4.1 thinks at most 32768 tokens when neither the req…
fork h4: STRATA_REASONING_BUDGET_TOKENS + STRATA_REASONING_LOOP_RECOV…
Revert "fork h4: STRATA_REASONING_BUDGET_TOKENS + STRATA_REASONING_LO…