Repository navigation
Conversation
…checkpoints llama_model_n_swa() returns 0 for LLM_ARCH_DEEPSEEK4 because the dsv4 cache's 128-token raw window cannot be used for rollback. LLM_ARCH_DEEPSEEK41 was added without extending that case, so llama-server computed its checkpoint threshold with n_swa = 128 and rejected every context checkpoint within 128 tokens of the resume point - including the one it creates 4 tokens before the end of the prompt. Each follow-up turn on a long conversation then resumed from the previous checkpoint, ~2048 tokens earlier. Measured on DeepSeek-V4.1-Flash Q3_K_M (12k-token document, four ~550-token follow-ups, temperature 0, RTX A6000 + RTX 3090 + 5975WX): turns 2-5, tokens re-processed: 2606/2587/2618/2579 -> 562/543/574/535 turns 2-5, time to first token: 20.6-21.3 s -> 8.2-8.9 s answers: byte-identical, 5/5 Trace before the fix (-lv 5): 'checking checkpoint with [11939, 11939] against 11816' (pos_next 11944 - n_swa 128) -> rejected, restored 9895. After the fix the same checkpoint is checked against 11944 and restored, as mainline does for DeepSeek-V4.
midagedev
added a commit
to midagedev/rig-log
that referenced
this pull request
Sep 14, 2026
…ecord row and dossier updated Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Author
|
Carried into ggml-org#28696 as 3b1179f. Thanks for picking it up. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
llama_model_n_swa()returns 0 forLLM_ARCH_DEEPSEEK4because the dsv4 cache's 128-token raw window cannot be used for rollback (the comment in the function says why).LLM_ARCH_DEEPSEEK41was added next to it without extending that case — every otherDEEPSEEK4special case insrc/(graph selection, KV cache construction, theis_dsv4checks inllama-arch.cppandllama-context.cpp) already lists both. This one line does the same here.Effect without it.
llama-servercomputes its checkpoint threshold aspos_next - n_swa, withn_swa = 128for V4.1, and rejects every context checkpoint within 128 tokens of the resume point — including the one it creates 4 tokens before the end of the prompt. Each follow-up turn on a long conversation resumes from the previous checkpoint, about 2 048 tokens earlier, and re-prefills that stretch.Measured on DeepSeek-V4.1-Flash Q3_K_M (12k-token document, then four ~550-token follow-ups,
temperature 0,cache_prompton; RTX A6000 + RTX 3090 + 5975WX,-c 262144 -ub 2048 -ctk q8_0 -ctv q8_0, DSpark draft):prompt_nruntime/deepseek41at 5210c7cTrace before (
-lv 5, turn 2):after:
Mainline master with DeepSeek-V4-Flash restores the same 4-tokens-before-the-end checkpoint on every turn (measured, 862 tokens per follow-up), so this brings V4.1 to the behaviour V4 already has.
The same symptom can be fixed on the server side by treating
pos_min == pos_maxcheckpoints as exact states (which is what ggml-org#25592 does for hybrid/recurrent memory); I tried that first and it measures the same, but for this model the model-side special case already exists and is the intended mechanism, so this is the smaller change.If this branch gets squashed before it goes upstream, I'd appreciate the commit (or a
Co-authored-by) surviving — no worries if the history has to be rewritten.Prepared with AI assistance (Claude); the measurements were run by me.