Repository navigation
bug: problematic cache reuse #164
Description
Bug
On moderate to long conversation, I noticed that the inference engine cache is not being used efficiently .
Turns becomes longer and longer, and backend metrics and logs confirms it.
Reproduction
Run a long multi-turn conversation and measure speed + check you backend logs, notice mostly full prompt reprocessing at every turn, including in between tool calls.
Likely Root cause
In prompt_templates, {{MEMORY}} and maybe others fields like {{FILE_TREE}} are recomputed and alterated at each turns.
It modifies the conversation history, and the backend is loosing most of its cache because of the early divergence.
Suggestions
These fields should probably be frozen for the current discussion, and if the Agent needs update:
- they can use tool call to retrieve updated values
- and / or the harness can inject updated value at the bottom of the conversation rather than alterating the prefix
Environment
- cptr 0.9.13
- llamacpp
_Disclosure: this issue was investigated and root-caused with the help of ChatGPT , and is written and submitted on my behalf at my request.
Activity
owui-terminator commentedon Jul 23, 2026
🔍 Related Issues Found
I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:
🟢 #162 bug: repeated context compaction can drop every user message and break Qwen/vLLM
Both issues concern long-running agent conversations getting worse over time because the message sequence sent to the backend changes in ways that hurt correctness/efficiency. Issue bug: repeated context compaction can drop every user message and break Qwen/vLLM #162 is about repeated compaction altering the retained conversation history; bug: problematic cache reuse #164 is about changing prompt template fields like MEMORY/FILE_TREE and losing cache reuse due to early prompt divergence.
by calderanicola
🟢 #13 feat: tool execution policies / allow once, always allow, "auto mode"
This feature request discusses an 'auto mode' task model that gets context from the conversation and is explicitly 'cache optimized for cheap task model cache hits.' That makes it relevant to bug: problematic cache reuse #164's concern about preserving stable prefixes and improving inference cache reuse across turns.
by Classic298
💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.
This comment was generated automatically. React with 👍 if helpful, 👎 if not.