Repository navigation
kv-grow: a run that cannot lend cache slots maps the whole window up front instead of writing past its first 16K cells - #1231
Open
BlueKingMuch wants to merge 1 commit into
Open
BlueKingMuch wants to merge 1 commit into
BlueKingMuch wants to merge 1 commit into
Conversation
… up front; its table uploads wait for the copy With --kv-grow the K/V is made elastic at session init and maps 16,384 cells. Growth needs more than init checks: a device residency table, cache slots and a loan above the floor. Without them (--no-pool, --no-token-graph, the dump modes, --expert-cache auto resolving to 0) kvg_start left the growth off, nothing mapped more, and a prompt past 16K wrote K/V into unmapped memory: an illegal address. That run now maps the whole window from new memory at start, or stops with a message when it does not fit. The growth's and the trim's residency-table uploads also wait for their own copy (a pageable cudaMemcpy may return before its DMA lands; the streams that read the table are non-blocking). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio
fragtion
pushed a commit
to fragtion/StrataPool
that referenced
this pull request
Oct 7, 2026
…ache slots maps the whole window up front; its table uploads wait for the copy Elastic K/V: a run that cannot lend cache slots maps the whole window up front; its table uploads wait for the copy With --kv-grow the K/V is made elastic at session init and maps 16,384 cells. Growth needs more than init checks: a device residency table, cache slots and a loan above the floor. Without them (--no-pool, --no-token-graph, the dump modes, --expert-cache auto resolving to 0) kvg_start left the growth off, nothing mapped more, and a prompt past 16K wrote K/V into unmapped memory: an illegal address. That run now maps the whole window from new memory at start, or stops with a message when it does not fit. The growth's and the trim's residency-table uploads also wait for their own copy (a pageable cudaMemcpy may return before its DMA lands; the streams that read the table are non-blocking). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0157fjixQD2nRxvCvtXq66DD
This was referenced Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With
--kv-growthe K/V is made elastic at session init and maps its first 16,384 cells (STRATA_KV_GROW_INIT).Growing needs more than that: a device residency table, cache slots and a loan above the floor of 128 slots.
When a run lacks one of them,
kvg_startleaves the growth off and nothing ever maps more, so a prompt past 16K writes K/V into unmapped memory.This is @sergqwer's fix from #378 (d5ee0cc), which the port into 0.1.40 does not have, cherry-picked onto 0.1.40.1 with him as the author:
cudaMemcpymay return before its DMA lands, and the streams that read the table are non-blocking).Reproduced
On 0.1.40.1 (82f46a8): RTX 4080 SUPER 32 GB, Windows 11, IQ3_S, 262K context, int8 K/V,
--kv-grow --expert-cache 80(102 cache slots, below the floor), one conversation of 30,000 tokens and then three turns of +2,300:prefill copy_i32: an illegal memory access was encounteredThe second part (the uploads) did not fail here; it is in the PR because it is part of the same fix in #378.
cc @sergqwer