Frequent thinking loops.

Does anyone else experience this? I'm seeing a whole lot of thinking loops occur, and i mean, literally the same block of text repeated over and over. It's not a rare event either, so far, maybe 1/4 - 1/3 of my prompts end in a loop. That's with standard settings, with and without "experimental-speed-projection".

I've been running 3.8 next for some time now, in llama.cpp, and i haven't seen it loop once until now.

I'm not sure if it's an issue with Strata or the quant though. I'm running ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S version. In llama.cpp, i used unsloth's UD_Q3_K_XL and UD_IQ4_XS, currently using AtomicChat's 4.27bpw one and this one, haven't had issues with any of them.

The loops seemed to happen mostly when i was asking to describe or ask a question about images. I'm trying regular textual prompts right now, they complete correctly more frequently, but still loop sometimes.

I also tried to use an RP system prompt, and the responses looked really odd, just a constant spam of "Need maybe ... need maybe ... need maybe ..." in the thought process, and the final answers also felt quite a bit different from what i was seeing from this model before. Though that was with experimental-speed-projection thingy turned on for refusal reduction, so maybe it is to blame. With no prompt, or without projection, thinking looks normal (even if it loops).

I'll try to check ISTA-DASLab's quant in llama.cpp later, see if the problem occurs there. But in the meantime, is there a way to load other quants in Strata?

are you using the qwen recommended thinking settings? maybe also see #53 , i saw too repetions in output without the recommended thinking setup - but they were also present in my llamacpp comparision (so not inference engine issue).

are you using the qwen recommended thinking settings?

Which settings? I don't know, i'm using whichever settings Strata sets by default. Honestly, i'm not even sure how to change sampling parameters or anything here, i don't see them in the config.

They are in the gear symbols in the webui chat - a side menu will open

These are the qwen recommended settings for thinkings:

{
  "temperature": 1.0,
  "top_p": 0.95,
  "top_k": 20,
  "min_p": 0.0,
  "presence_penalty": 0.0,
  "frequency_penalty": 0.0,
  "repetition_penalty": 1.0,
}

And if youre using the api then inside the /v1/chat/completions

Well, maybe the issue is with the quant, hard to say. I tried to load it up in llama.cpp, asked the same question about the same image, which sent Strata into loops multiple times, and although in llama.cpp it didn't loop exactly, it did spent 40k tokens thinking about more or less the same things, over and over. Which is also not something i had happen with aforementioned quants, at least not for so long, and not on a simple question like that.

Two things I confirmed in the engine that may matter for the image-prompt loops, since image requests only ever take the text path's settings today:

A request with an image decodes greedy, always. serve/server.py:279 drops the sampling keys whenever a request carries embeddings, and the engine's GENI parser skips everything but cvec= (src/program/generate.cpp:2651). So temperature / top_p / top_k on a vision request are accepted and then silently discarded — no warning, no ERR.
That also means the recommended Qwen thinking settings cannot be applied to image requests at all, which is consistent with the loops showing up mainly there.

Not claiming it is the cause — the IQ3_S quant is still an untested suspect — but the fix on the engine side is small: parse and honor the same sampling keys on GENI, or answer ERR naming the unsupported field so it is never silently ignored.

For reference, I measured the throughput side of the recommended set on the IQ1_M Coder build (RTX 5090, 256-token code reply, engine-side numbers from the log): greedy temperature=0 runs 118-126 tok/s at 72-75% draft acceptance; the Qwen thinking set (temperature=1.0, top_p=0.95, top_k=20) runs 84-98 tok/s at 63-77%. So the sampled path costs roughly 25-30% and the MTP acceptance rate barely moves — the cost is the sampler's extra passes over the 40,525-token head, not speculative decoding.

Config tip if you did not find it: sampling defaults go in a "sampling" block in the run config JSON passed to serve/server.py --config, not in anything else. The engine's own CLI flags (--top-k/--top-p/--temperature only) do not reach the server path at all.

Well, i've spent some 100k+ tokens throwing various pictures at Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S in llama.cpp, and i don't observe loops so far. A lot of typical 3.8 overthinking, going over the same "thoughts" repeatedly, maybe somewhat more than it happened with the other quants i tried, but not literally looping. So i'm inclined to say that it is something with the engine after all.

try this jinja template it solved overthinking and loops for me. I use it also with the other Qwen models. Its really a night and day difference: #99

@jleaders

Found one real cause on our side, thanks to @jleaders: a request with an image always decoded greedily - the server sent only the speed-projection key with image requests and the engine skipped every other sampling setting for them, so the recommended thinking settings (temperature 1.0, top_p 0.95, top_k 20) never reached image questions. Fixed on the 0.1.23 branch: image requests take the same sampling settings as text. Until then, for text requests the settings in the chat's gear menu (or a "sampling" block in the run config for API clients) apply. I'll post here when 0.1.23 is out.

Image requests honor sampling (temperature, top_p, top_k, penalties, …

Fixed in 0.1.23 (released): image requests now use the same sampling settings as text (temperature, top_p, top_k, penalties, seed) - they always decoded greedily before, which fits the loops you saw on pictures. Run START-HERE.bat to update. If loops still happen with the recommended thinking settings, please reopen with the prompt.

Hola, hice una prueba, le pedí que buscara una cita textual y entro en loop intentando inventar la respuesta, en strata el Pensamiento entró en bucle infinito, les dejo un extracto:

Busca en Wikipedia en español el artículo sobre bomba centrífuga y encuentra una explicación

Entrégame una cita textual exacta de 25 a 50 palabras, indica la sección donde aparece y añade el enlace directo a la página
You · 12:16 a. m.
Thinking…
We need answer in Spanish. User asks: "Busca en Wikipedia en español el artículo sobre bomba centrífuga y encuentra una explicación

Entrégame una cita textual exacta de 25 a 50 palabras, indica la sección donde aparece y añade el enlace directo a la página"

We need provide exact quote 25-50 words from Spanish Wikipedia article "bomba centrífuga", indicate section, direct link. We don't have browsing. Need maybe use knowledge? But user asks search. We as model maybe can't browse unless tool. However developer says "You are an AI assistant accessed via an API. Tools available?" No browsing tool listed. Need answer from memory? Must be exact textual quote. Could be risky. Need maybe know article es.wikipedia.org/wiki/Bomba_centrífuga. Need exact quote of 25-50 words. We need be truthful. If cannot verify exact quote, maybe state I cannot access internet/verify? But they ask specific. We should maybe use general knowledge or known article. Need maybe provide citation if know. But exact text? Let's think of Spanish Wikipedia article on bomba centrífuga. It likely starts: "Una bomba centrífuga, también llamada bomba turbo, es una máquina hidráulica que convierte energía cinética en energía potencial mediante un impulsor o rodete que gira..." Hmm. Need exact 25-50 words. Need know article section. Could use Wikipedia API if we had internal? But no browse tool. We may simulate? Need maybe call no tool. We need perhaps produce answer with caveat. But final must satisfy? Maybe we can reconstruct from Spanish Wikipedia. Let's search memory? The article "Bomba centrífuga" on es.wikipedia. Lead: "Una bomba centrífuga o bomba turbo es una máquina hidráulica que convierte energía cinética en energía potencial mediante un impulsor o rodete que gira, incrementando la velocidad del líquido y dirigiéndolo hacia un difusor o voluta que lo desacelera, aumentando la presión." That is one sentence? Count words maybe 42. Let's count: Una(1) bomba2 centrífuga3 o4 bomba5 turbo6 es7 una8 máquina9 hidráulica10 que11 convierte12 energía13 cinética14 en15 energía16 potencial17 mediante18 un19 impulsor20 o21 rodete22 que23 gira24 incrementando25 la26 velocidad27 del28 líquido29 y30 dirigiéndolo31 hacia32 un33 difusor34 o35 voluta36 que37 lo38 desacelera39 aumentando40 la41 presión42. Good 42. But is that exact? Maybe article says "Una bomba centrífuga, también llamada bomba turbo, es una máquina hidráulica..." Could be with commas. Need exact? Maybe the current lead: "Una bomba centrífuga es una máquina hidráulica que convierte energía cinética en energía potencial mediante un impulsor o rodete que gira..." Hmm.

Need section: probably "Descripción" or "Principio de funcionamiento"? Article may have sections: Historia, Principio de funcionamiento, Tipos, Aplicaciones, Partes, Funcionamiento, Curvas características, etc. A lead quote doesn't have section? Could be "Introducción" not section? User asks indicate section where appears. Better choose a section quote. Need exact 25-50 words.

Could use a known quote from section "Principio de funcionamiento"? Let's infer. Spanish Wikipedia: "Bomba centrífuga" section "Principio de funcionamiento": "El fluido entra axialmente por el ojo del impulsor y es acelerado radialmente por las palas del rodete, aumentando su velocidad y presión. La voluta o difusor convierte parte de la energía cinética en energía de presión." Count maybe 40. But exact?

Maybe article says: "En una bomba centrífuga, el líquido entra por el centro del impulsor, llamado ojo, y es acelerado por las palas del rodete. Este aumento de velocidad se transforma en presión al salir hacia la voluta o difusor." Not exact.

Need maybe rely on current Wikipedia content from memory. Could use API in mind? No.

Could answer: "No puedo acceder en tiempo real a Wikipedia desde esta sesión para garantizar una cita textual exacta. No, user wants exact quote. We need maybe do web? But tool not provided. In this environment, maybe expected to answer with no live browsing, but maybe can provide a quote from memory if confident. The question likely tests tool use? There is no browse. We need be honest.

Maybe we can access Wikipedia via hidden? The instruction: "Busca en Wikipedia en español..." Could expect us to use tool? But no browsing tool. Some models may have browsing via internal? We don't.

Need satisfy: exact quote 25-50 words. Could choose quote from article and note from memory, maybe not guaranteed. Need maybe direct link. The link: https://es.wikipedia.org/wiki/Bomba_centr%C3%ADfuga (direct page). Section maybe "Descripción". Need exact quote. Maybe use the lead's definition: "Una bomba centrífuga es una máquina hidráulica que se vale de un rotor o impulsor para mover fluidos y aumentar la presión y la velocidad de los mismos." Count 23? Una1 bomba2 centrífuga3 es4 una5 máquina6 hidráulica7 que8 se9 vale10 de11 un12 rotor13 o14 impulsor15 para16 mover17 fluidos18 y19 aumentar20 la21 presión22 y23 la24 velocidad25 de26 los27 mismos28. Good 28. But exact? I recall a definition: "Una bomba centrífuga es una máquina hidráulica que se vale de un rotor o impulsor para mover fluidos y aumentar la presión y la velocidad de los mismos." This might be from "Bomba centrífuga" article? Let's search memory: It might be from a textbook, maybe Wikipedia has "Una bomba centrífuga es una máquina hidráulica que se vale de un rotor o impulsor para mover fluidos y aumentar la presión y la velocidad de los mismos." I remember from es.wikipedia? It might be exact. Count 28. Need 25-50. Section: perhaps "Descripción" or lead (intro). Could say appears at beginning in section "Descripción" (if article starts with that). But exact? Maybe article has first section "Descripción". Let's check known es article structure: For pump: "Descripción" likely. The quote could be:

En llama cpp, el resultado fue diferente, concluyó rápido:

Busca en Wikipedia en español el artículo sobre bomba centrífuga y encuentra una explicación

Entrégame una cita textual exacta de 25 a 50 palabras, indica la sección donde aparece y añade el enlace directo a la página

382 tokens
13s
28.22 tokens/s

Reasoning
Toggle content
The user is asking to search for an article about "bomba centrífuga (centrifugal pump)" on the Spanish Wikipedia, find the description, and present a quotation of 25-50 words along with the section name and a direct link.

I don't have a web browsing tool. The only available tools are get_datetime and get_info. In other words, I can't actually fetch Wikipedia. I should honestly convey that I can't browse the web. However, I might be able to recite from memory what's written in the Spanish Wikipedia article on "bomba centrífuga." What the user wants is "an exact verbatim quotation," and there's a risk of fabrication. In fact, I do remember the Spanish Wikipedia article "Bomba centrífuga." The introductory section usually reads something like: "Una bomba centrífuga es una máquina de fluido que convierte la energía cinética en energía potencial mediante un impulsor que gira a alta velocidad (High.)" Hmm, but I can't verify it.

Policy: Don't fabricate quotations. I should be honest and say that I can't access Wikipedia because I don't have internet access. I can offer the link https://es.wikipedia.org/wiki/Bomba_centrífuga and a paraphrase from my knowledge, but I should clearly state that it's not a verified, verbatim quotation.

Should I still try calling get_info? Useless. I'll just honestly say that I can't browse the web.

I'll provide a cautious answer: I'll offer the URL and explain the typical explanation, but note that it may not be accurate.

No tengo acceso a internet ni herramientas de navegación en este entorno, por lo que no puedo consultar Wikipedia en tiempo real ni verificar una cita textual exacta. Darte una "cita exacta" de memoria sería arriesgado, porque podría no coincidir palabra por palabra con el texto actual del artículo.

\R

Lo que sí puedo ofrecerte de forma fiable:

Enlace directo al artículo:
https://es.wikipedia.org/wiki/Bomba_centr%C3%ADfuga

Explicación general (parafraseada, no textual):
Una bomba centrífuga es una máquina de fluidos que transforma energía cinética en energía de presión. Un impulsor (rodete) que gira dentro de una carcasa acelera el fluido hacia el exterior por fuerza centrífuga, aumentando su velocidad y, al diffundiarse en la voluta o los canales difusores, esa velocidad se convierte en presión, lo que permite elevar o transportar líquidos.

Esta descripción corresponde típicamente a la sección introductoria (lead/encabezamiento) y a la sección de principio de funcionamiento del artículo.

Recomendación: si necesitas la cita literal de 25–50 palabras con la sección exacta, abre el enlace anterior y busca el primer párrafo del artículo; ahí suele encontrarse la definición formal. Si puedes pegarme el texto del párrafo, con gusto selecciono y verifico la cita exacta que cumpla el límite de palabras que pides.

qwen3.8-gsq-flash-iq3_xxs-qsa-direct
657 tokens
33s
19.75 t/s

Image requests honor sampling (temperature, top_p, top_k, penalties, …

Additional observation on v0.1.38: repeated client-executed tool calls across model requests, despite an existing per-request thinking budget. I am adding this here because the maintainer routed this kind of tool-loop evidence to #75 in #602.

Observed behavior

During one text-only code/document review, the model drafted a JSONL report, then repeatedly attempted the same ineffective JSON repair:

98 consecutive identical shell commands, spanning 2,091.936 seconds (about 34.9 minutes).
At an intermediate checkpoint, the draft had 118 lines and the same five lines still failed JSON parsing. The repair command returned exit code 0; successful tool execution did not mean the intended repair had succeeded.
At minute 55, the harness delivered a message requiring the model to consolidate the report and finish within the remaining five minutes. Delivery was confirmed in the original session.
After that message, the model did repair the JSON syntax, but continued rereading source files instead of completing the task.
At minute 60, the harness interrupted the active turn. The native task ended as incomplete; total reviewer elapsed time was 3,600.738 seconds. The syntactically valid draft was retained and was not treated as a completed delivery.

This was continued model/tool activity, rather than an engine that stopped emitting tokens. We did not repair the draft ourselves or rerun the task to replace the failed observation.

Environment and relevant settings

Strata v0.1.38, official prebuilt NVIDIA engine; also the latest published release when checked for this report.
Qwen3.8-Flash-Next GSQ-RCO Q2_0, text only.
Inference on Windows native Python/CUDA; Codex 0.160.0 on WSL owns the session and executes shell/file tools.
Hardware: i9-14900K, 64 GB RAM, RTX 4070 Ti SUPER 16 GB.
Codex reasoning effort: medium.
Per-request defaults for that effort: temperature=0, top_p=1, top_k=20, reasoning_budget_tokens=4096, max_tokens=16384.
Speculative decoding setting: --spec-min-p 0.7 (this is not the sampling min_p setting).
A separately trained expert-cache profile was loaded, with saving disabled during evaluation. This changes expert placement/profile data, not model weights. It is a relevant condition, not an identified cause; no matched default-profile reproduction of this loop has been established.
Codex Responses requests pass through our local adapter to Strata Chat Completions. Ordinary tool calls are returned to Codex and executed by its harness. This observation has not been isolated to a direct-Strata minimal reproducer, so the adapter/harness remains part of the reproduction conditions.

Reproduction scope and distinction from other reports

This is one directly observed failed instance, not a measured failure rate or a deterministic minimal reproducer. The original task was a substantial multi-file review requiring a structured report and validation. Full private session logs and task contents are not attached; the counts above were checked against those logs and preserved drafts.

The behavior resembles the repeated verification/tool calls reported in #602. It differs from #606's persistent single-character output on subsequent independent requests, and from #702's repeated sentences/thinking in ordinary chat. We have not established a shared root cause. Since this run used greedy sampling, it also does not establish whether the recommended sampled thinking settings prevent this behavior.

Questions for mitigation

Does Strata have a supported repeated-tool/no-progress guard for client-owned tool loops across requests, or a supported configuration we have missed? The v0.1.38 engine-silence watchdog and server-owned MCP max_rounds limit appear to address different boundaries.
Is there a recommended inference configuration or model/quantization comparison to distinguish model behavior from an inference-path problem? We would prefer a controlled comparison to assuming that token-level repetition penalties solve semantic repetition.
Would an optional diagnostic/guard be appropriate, or should this live entirely in the calling harness? A per-request thinking/output cap was already enabled here, but the next request receives a fresh budget. Our current fallback is a task deadline; possible earlier intervention is to detect repeated identical calls with unchanged failure evidence, send one warning, then interrupt if no recovery occurs.

A warning produced partial recovery in this instance, but did not produce completed delivery. We are therefore reporting both the recovery and the remaining failure, without claiming that reminders alone prevent recurrence.

Following the maintainer's request in #602, here are the raw artifacts from our local runs:

https://github.com/zimuhuan-code/dsh-local-loop-fuse/tree/main/evidence

What it is: three captures from two sessions, written by our harness-side guard at the
moment it detected a repeated tool call — the same bash call with identical arguments six times
in a row (consecutive: 6), with every matching tool/result returning success. Strata 0.1.37
(db4f91a), Qwen3.8-Flash-Next IQ3_S, 2× RTX 3060, 2026-10-03 14:10–14:57 local. Plus the
guard's cancellation log and activity log for those sessions.

Context that may matter: both sessions were doing the same kind of work — reviewing a draft bug
report about the engine, checking its claims one at a time against local notes. In both, the loop
started while re-checking a single fact (whether a long-context section of a note contained an
IQ3_S entry), and the repeated command is a two-part grep/awk whose output never reaches a
terminal "confirmed" state. In the second session the loop never produced a review at all: the
harness injected its repeated-call warning twice (consecutive_calls: 5) and the operator stopped
the session by hand.

A correction to our own #602 report — please read this part. Our earlier "we have reproduced
issue #75 locally" claim was partly wrong. We verified every capture we had against the
model field recorded in its source session, and five of the eight came from a session running a
hosted API model (provider: deepseek-official, model: deepseek-flash) — not Strata. Those
are removed, and we specifically withdraw the two observations that had been attached to them:

that a stop message does not end the loop; and
that the model re-issues the same call in the next turn.

What the three remaining captures support is the narrower statement above. We would rather correct
this ourselves than have it discovered later.

Still not claiming: no rate, no minimal reproducer, no mechanism inside the engine — three
captures, two sessions, one machine, one quantisation, text-only prompts, no images. Happy to run a
controlled A/B prompt set if that would be more useful than captures.

Related: our benchmark report for the same configuration (IQ3_S, 2× RTX 3060,
Strata 0.1.37) is now open as a PR — #721 — results only, no engine changes.

It loops a few times, I mostly ask for markdown output with Linux commands for example and it loops after a while, even during thinking. I set thinking to low to avoid that but I think it did that with low also.

Image requests honor sampling (temperature, top_p, top_k, penalties, …

Navigation Menu

Repository navigation

Frequent thinking loops. #75

Observed behavior

Environment and relevant settings

Reproduction scope and distinction from other reports

Questions for mitigation

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Frequent thinking loops. #75

Description

Activity

hendrikp commented on Sep 28, 2026

izMoff commented on Sep 28, 2026

hendrikp commented on Sep 28, 2026

izMoff commented on Sep 28, 2026

jleaders commented on Sep 28, 2026

izMoff commented on Sep 28, 2026

technerd-official commented on Sep 29, 2026

Niko1221 commented on Sep 29, 2026

Niko1221 commented on Sep 29, 2026

VacunaHumilde commented on Sep 30, 2026

pellorao commented on Oct 4, 2026

Observed behavior

Environment and relevant settings

Reproduction scope and distinction from other reports

Questions for mitigation

zimuhuan-code commented on Oct 4, 2026

linuxtuga commented on Oct 4, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions