Skip to content

deepseek41 : report n_swa = 0 like deepseek4 so the server keeps its checkpoints - #3

Closed
midagedev wants to merge 1 commit into
vcruz305:runtime/deepseek41from
midagedev:fix/deepseek41-n-swa
Closed

midagedev wants to merge 1 commit into
vcruz305:runtime/deepseek41from
midagedev:fix/deepseek41-n-swa

Conversation

@midagedev

Copy link
Copy Markdown

llama_model_n_swa() returns 0 for LLM_ARCH_DEEPSEEK4 because the dsv4 cache's 128-token raw window cannot be used for rollback (the comment in the function says why). LLM_ARCH_DEEPSEEK41 was added next to it without extending that case — every other DEEPSEEK4 special case in src/ (graph selection, KV cache construction, the is_dsv4 checks in llama-arch.cpp and llama-context.cpp) already lists both. This one line does the same here.

Effect without it. llama-server computes its checkpoint threshold as pos_next - n_swa, with n_swa = 128 for V4.1, and rejects every context checkpoint within 128 tokens of the resume point — including the one it creates 4 tokens before the end of the prompt. Each follow-up turn on a long conversation resumes from the previous checkpoint, about 2 048 tokens earlier, and re-prefills that stretch.

Measured on DeepSeek-V4.1-Flash Q3_K_M (12k-token document, then four ~550-token follow-ups, temperature 0, cache_prompt on; RTX A6000 + RTX 3090 + 5975WX, -c 262144 -ub 2048 -ctk q8_0 -ctv q8_0, DSpark draft):

turns 2–5 prompt_n turns 2–5 TTFT answers
runtime/deepseek41 at 5210c7c 2606 / 2587 / 2618 / 2579 20.6 / 21.1 / 21.3 / 21.1 s —
+ this commit 562 / 543 / 574 / 535 8.9 / 8.8 / 8.8 / 8.2 s 5/5 byte-identical

Trace before (-lv 5, turn 2):

checking checkpoint with [11939, 11939] against 11816   <- 11944 - 128, rejected
checking checkpoint with [9895, 9895] against 11816     <- restored

after:

checking checkpoint with [11939, 11939] against 11944   <- restored

Mainline master with DeepSeek-V4-Flash restores the same 4-tokens-before-the-end checkpoint on every turn (measured, 862 tokens per follow-up), so this brings V4.1 to the behaviour V4 already has.

The same symptom can be fixed on the server side by treating pos_min == pos_max checkpoints as exact states (which is what ggml-org#25592 does for hybrid/recurrent memory); I tried that first and it measures the same, but for this model the model-side special case already exists and is the intended mechanism, so this is the smaller change.

If this branch gets squashed before it goes upstream, I'd appreciate the commit (or a Co-authored-by) surviving — no worries if the history has to be rewritten.

Prepared with AI assistance (Claude); the measurements were run by me.

…checkpoints

llama_model_n_swa() returns 0 for LLM_ARCH_DEEPSEEK4 because the dsv4 cache's
128-token raw window cannot be used for rollback. LLM_ARCH_DEEPSEEK41 was added
without extending that case, so llama-server computed its checkpoint threshold
with n_swa = 128 and rejected every context checkpoint within 128 tokens of the
resume point - including the one it creates 4 tokens before the end of the
prompt. Each follow-up turn on a long conversation then resumed from the
previous checkpoint, ~2048 tokens earlier.

Measured on DeepSeek-V4.1-Flash Q3_K_M (12k-token document, four ~550-token
follow-ups, temperature 0, RTX A6000 + RTX 3090 + 5975WX):

  turns 2-5, tokens re-processed: 2606/2587/2618/2579 -> 562/543/574/535
  turns 2-5, time to first token:  20.6-21.3 s -> 8.2-8.9 s
  answers: byte-identical, 5/5

Trace before the fix (-lv 5): 'checking checkpoint with [11939, 11939] against
11816' (pos_next 11944 - n_swa 128) -> rejected, restored 9895. After the fix
the same checkpoint is checked against 11944 and restored, as mainline does
for DeepSeek-V4.
midagedev added a commit to midagedev/rig-log that referenced this pull request Sep 14, 2026
…ecord row and dossier updated

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@midagedev

Copy link
Copy Markdown
Author

Carried into ggml-org#28696 as 3b1179f. Thanks for picking it up.

@midagedev midagedev closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant