Infinite loop issue

Having switch-1.50iq2_xs, 8 bit KV cache, 128k context size, engine version v0.1.38 running out of the box after install on 4090 and 64 GB RAM, Windows 11.

Tried OpenCode and Qwen Coder but both will enter endless loop after a while.
Performance is good at 103 tok/s except when this issue was hit.

Is there anything I can tweak to prevent this?

Most likely this is related to model settings and not the engine it self.
When a reasoning model gets stuck in a loop, it means the token probabilities have collapsed.
change the temperature and top_p values for your model.
Lower temperatures in LLMs actually make them more prone to looping.

@tarunpaul

@tarunpaul That is a good point and I also suspected so. But I was using the default setting right after running setup.
My understanding is that Strata is particularly optimized for Qwen Flash Next and everything should already be in place nicely? No?

Also do you have a recommended setting? From the very limited docs I saw these default:
"sampling": {"temperature": 1.0, "top_p": 0.95, "top_k": 20}

Correct me if I am wrong.

@davidliudev

@davidliudev Since you mentioned that you’re using it with OpenCode and Qwen Coder, it might be worth checking whether either of them is overriding the model’s sampling settings. That could potentially explain the looping behavior.

@tarunpaul

@tarunpaul I just double checked and the default setting is actually indeed the official qwen flash next recommended settings.

Found a reddit post and someone suggested adding presence_penalty of 1.5. That is originally supposed to be only for the non-thinking mode, but considering the quantization loss I enabled it with thinking mode anyway.

I then updated strata-swift-iq2_xs.json and add that to it.

So far it works for a while without the looping issue anymore. If it comes back I will update this thread.

@davidliudev

@davidliudev Glad to hear that the presence penalty seems to be helping. I'll be interested to hear if you experience the looping issue again.

It might also be worth taking a look at the issue below, as it describes a somewhat similar looping behavior with Qwen Flash Next and could provide some additional clues:

#75

I had the same issue with swift1.5-iq2_xs and tried a few things. But the issue was completely solved, when switching to standard (non-swift) iq3_xxs. So I think, it's more a general behaviour of swift. For coding it worked good for me, but not much else. With standard iq3_xxs, I wrote even a 300 page story without any repetition.

Just report back that it still happen, but much less frequently so at least it is usable for now. I would just treat that as a temporary workaround and keep this issue open.

@RobinMeyer

I had the same issue with swift1.5-iq2_xs and tried a few things. But the issue was completely solved, when switching to standard (non-swift) iq3_xxs. So I think, it's more a general behaviour of swift. For coding it worked good for me, but not much else. With standard iq3_xxs, I wrote even a 300 page story without any repetition.

@RobinMeyer Do you mind sharing what is your setup? I am wondering if I should also switch to it. Not sure if my setup can handle iq3 though.

@davidliudev

@davidliudev :
Because of Strata, I updated extra to 64GB RAM and I have also a RTX5070ti and a 7950x3d.
At first, I used Swift1.5-iq2_xs with 32GB RAM, which worked well with paging with 40-80t/s.
But according the docu, 12GB VRAM and 64GB RAM is sufficient. Perhaps it works with paging with 48GB RAM. RAM is filled with 53GB.

Another data point, from the Coder (not Swift), with what looks like a specific trigger and a tested way to bound it.

Setup: Coder IQ1_M, engine 0.1.39, Tesla V100 32 GB (low-RAM mode), default sampling from setup, reached through
/v1/messages by a coding agent (Claude Agent SDK, output_config.effort medium).

What happened: in 1 of 20 agent jobs, one thinking block ran to 140,267 characters and stopped only at the client's
32K output cap (about 5 minutes on this card). It is a loop of a single sentence, repeated 1,110 times:

Actually, wait: maybe the screenshot is showing the app's old Disk bar? No, the screenshot shows both Disk and Steam rows.

The 256-repeated-token guard (#606) does not catch this, since the repeat is a whole sentence.

Likely trigger: the block opens with the model puzzling over numbers "in the desktop screenshot" that contradict the
data it had just read from the app's web page. It never received that screenshot: images inside a tool_result were being
dropped (that's #819), so it reasoned about a picture it had imagined, met a contradiction it could not resolve, and
circled. That may not be the cause of every loop here, but "the model reasons about input it cannot see" seems worth
knowing. Since applying #819 we have run two agent jobs without a recurrence -- far too few to say more.

Bounding it: a reasoning_budget_tokens cap. On a prompt built to make it think at length (effort high, max_tokens
6000): no cap -> ~3.4K tokens of thinking until finish_reason: length and no answer; reasoning_budget_tokens: 400 ->
the server's wrap-up line ("I have thought about this long enough; time to give my answer.") and a full answer, 28 s
instead of 75 s. In our agent runs the longest normal thought was ~4K tokens, so we now run with
"reasoning_budget_tokens": 8192 in strata-<model>.json. In the two agent jobs since (16K and 8K caps) it never fired,
and it would cut a loop like the one above to roughly 8K tokens. It bounds the damage; it does not stop the loop from starting.

Drafted with an AI assistant (Claude) from our logs and tests, and checked by me.

Another data point from the original Qwen3.8-Flash-Next model. I was able to trigger this with a very simple standalone coding prompt, without an agent, tools, images, MCP, or external context.

Setup:
Strata 0.1.39
Qwen3.8-Flash-Next IQ3_XXS
RTX 5080 16 GB
Ryzen 9 5900X
64 GB RAM
Windows
64K context, INT8 KV, vision disabled
temperature 0.60, top_p 0.95, top_k 20
reasoning effort: High

The test prompt was a small PowerShell task: write a function that performs a TCP reachability check with a maximum 5 second timeout and returns only a boolean result. Nothing unusual in the context.

I repeated the same prompt in fresh chats and got the following results:

Configuration	Output tokens	Time	Final answer
default (`presence_penalty=0`, no reasoning budget)	65,382	738.2 s	No - context exhausted
`presence_penalty=1.0`, no reasoning budget	42,000	539.3 s	Yes
`presence_penalty=1.5`, no reasoning budget	21,643	286.4 s	Yes
`presence_penalty=1.0` + `reasoning_budget_tokens=8192`	8,998	113.2 s	Yes

The final answers from all completed runs were technically usable; in particular, adding the 8192-token reasoning budget did not produce an obvious quality regression in this test.

I also fully restarted Strata and retried the same prompt with Thinking disabled. It still went beyond 17K generated tokens without reaching the final answer before I stopped it, so in my case this does not appear to be limited to reasoning_effort=high.

A second unrelated Home Assistant/state-machine design prompt completed successfully with the current workaround (presence_penalty=1.0, reasoning_budget_tokens=8192): 12,397 output tokens in 165.4 s, and the result was coherent and useful.

My current workaround is therefore:

"sampling": {
  "presence_penalty": 1.0
},
"reasoning_budget_tokens": 8192

This makes the model usable for me, but I see both settings as mitigation rather than a real fix. presence_penalty changes sampling behavior, while the reasoning budget only bounds the damage once a loop has already started.

The recent report about a loop triggered by reasoning over a missing image is very interesting, but my reproducible case contains no image or tool result at all, so there appear to be additional triggers.

It might be useful to detect repetition over longer token sequences / repeated n-grams or sentences inside the thinking block, rather than only identical-token repetition, and then use the same wrap-up path as reasoning_budget_tokens.

I can provide the exact prompt and logs if that would help reproduce it.

@MicEs1

Two data points that may explain why loops come back even with the "official" settings in strata-<model>.json.

1. The web chat sends its own temperature, 0.6 by default. In serve/web/app.js, DEFAULTS = {thinking: "high", temperature: 0.6, top_p: 0.95, top_k: 20, ...}, and every chat request sends temperature, top_p and top_k explicitly. Request values win over the config's sampling, so the browser chat runs at 0.6, not the 1.0 that setup writes (1.0 / 0.95 / 20 / presence 0 is the published Qwen3.8 thinking default). With "Use for other apps too" on, the chat's settings are also applied to API clients for every field they don't send. So an agent can end up at 0.6 as well. @MicEs1's loop above was at temperature 0.60.

2. Agents often send their own sampling, sometimes per task. In our setup (Hermes Agent, Qwen3.8-Flash-Next IQ3_S, later UD-IQ4_XS), the loops came from:

No sampling at all: neither the client nor the config set it, so the engine default, greedy, applied. One reply reached 102,833 tokens.
A side task the client sends at temperature 0.1: its image-description calls.

After putting the official values in the config and raising that side task to 1.0, we saw 0 runaway loops in the following ~19.5 h of agent work (#980). "reasoning_budget_tokens" (we use 16000) also bounds the worst case.

My suggestion (opinion): check what each request really carries. "api_monitor": true in strata-<model>.json, then http://127.0.0.1:<port>/api-monitor, shows every request's body. If you use the web chat, set its temperature to 1.0 under the sampling settings. If you set an API key, add it there too (About > Settings).

@talisp

Following up on @talisp's point about requests that carry no sampling: we checked ours, and then compared greedy against
the Qwen3.8 thinking defaults on real agent work.

What our client sends: a coding agent built on the Claude Agent SDK, talking to /v1/messages. Logging its requests
showed "temperature": null (no top_p / top_k either), only "thinking": {"type": "adaptive"} and
"output_config": {"effort": "medium"}. Our strata-coder-iq1_m.json had no sampling block, so every agent request
ran greedy. Our one loop earlier in this thread (the sentence repeated 1,110 times) happened under those settings.

The comparison: Coder IQ1_M, engine 0.1.39, Tesla V100 32 GB, reasoning_budget_tokens: 8192 in both. The second
config only adds "sampling": {"temperature": 1.0, "top_p": 0.95, "top_k": 20, "presence_penalty": 0.0}. Same five
agent jobs on each: four real coding tasks in an existing Python project, graded by hidden acceptance tests the agent
never sees, plus one six-phase web app built from scratch.

Job	Greedy (2 runs)	1.0 / 0.95 / 20 (1 run)
Task 1	pass, 6.3 / 6.7 min	pass, 6.5 min
Task 2	pass, 26.3 / 6.5 min	pass, 8.2 min
Task 3	pass, 7.2 / 8.9 min	pass, 8.6 min
Task 4	pass, 5.4 / 5.2 min	pass, 6.3 min
Six-phase app	done, 24 / 40 min	done, 47.5 min

No runaway thinking in any of the 15 jobs, at either setting.
Same results on the hidden tests. On the single tasks the times overlap; on the six-phase build the sampled run
thought noticeably more (prompts up to 97K vs 60K) and took longer. That is one run, so treat the time as a hint.
(The 40 min greedy run lost about 12 min to an unrelated problem in our own tooling.)
We kept greedy plus the 8K reasoning budget for now, since it already bounds a loop to about 8K tokens. That is a
choice for our setup, not a general recommendation: 15 jobs are far too few to measure a roughly 1-in-20 event.

The part that seems worth knowing for others: an agent that sends no temperature gets greedy unless the config has a
sampling block, and nothing in the client shows it. "api_monitor": true or a logging proxy makes it visible.

Drafted with an AI assistant (Claude) from our runs and logs, and checked by me.

serve: opt-in recovery from repeated reasoning passages

Fixed in 0.1.40: an opt-in repeat stop for reasoning that repeats whole passages: "reasoning_loop_recovery": "stop" ends the reply there, "recover" goes on with the low-effort instruction (#869). Off by default. See DETAILS.

Navigation Menu

Repository navigation

Infinite loop issue #728

27 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Infinite loop issue #728

Description

Activity

tarunpaul commented on Oct 4, 2026

davidliudev commented on Oct 4, 2026

tarunpaul commented on Oct 4, 2026

davidliudev commented on Oct 4, 2026

tarunpaul commented on Oct 4, 2026

RobinMeyer commented on Oct 4, 2026

davidliudev commented on Oct 4, 2026

davidliudev commented on Oct 4, 2026

RobinMeyer commented on Oct 4, 2026

christopherrobertbrooks-tech commented on Oct 4, 2026

MicEs1 commented on Oct 5, 2026

talisp commented on Oct 5, 2026

christopherrobertbrooks-tech commented on Oct 5, 2026

Niko1221 commented on Oct 6, 2026

27 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions