[BlueKingMuch]

Builds on #1163 (the first commit); this PR is the second.

An agent harness that runs subagents sends several conversations that start with the same system prompt and tool list. The conversation cache checkpoints that prefix, and when a new conversation finds it only inside a parked one (because another conversation is in the session at that moment), it moves that whole parked conversation into the session and overwrites it after the checkpoint. The parked conversation is gone, and its next turn reads its history again.

With this, the new conversation borrows: only the K/V up to the checkpoint and the checkpoint itself come over (conversation_snapshot_restore_prefix), and the parked conversation stays where it is. A match on a parked conversation's live state or its newest checkpoint is that conversation going on, and it is moved into the session as before (ConversationCache::borrows). While the outgoing conversation is parked, the one borrowed from is pinned, so parking cannot evict it; if the outgoing conversation fits only in its room, it is moved in as before and the outgoing one keeps its place. The engine log says "borrowed".

#1090 (prefix snapshots on disk) changes the same decision: a saved prefix wins over a parked conversation that matches no further, so that conversation is not taken out. With this PR it is not taken out either, and nothing is read from disk.

Measured

Same setup as #1163, also on 0.1.40.1 against v0.1.40.1 (a layer split never borrows - its stages restore whole images only - a borrow clears a batch slot as a source, as a taken conversation does, and without --mtp it takes no draft K/V, as parking saves none): RTX 4080 SUPER 32 GB, Ryzen 7 5800X3D, 96 GB, Windows 11, IQ3_S, 262K context, --conversation-cache-mib 2816 --conversation-cache-slots 3, a fixed expert placement (--adapt-every 1000000 --expert-cache 8031), greedy, 64-token answers. A main agent P (30,000 tokens) and two subagents S1 and S2 with the same system prompt (15,000 each) take turns for two rounds, every return +2,300 tokens: P1 S1 P2 S2 S1b S2b P3 S1c S2c P4. Prompt tokens read, time to first token:

	v0.1.40.1	#1163	#1163 + this PR
S2, new subagent while P is in the session	moves S1 in, S1 is overwritten	the same	borrows from S1 (32 ms), S1 stays parked
S1b, subagent 1 goes on	14,878, 4.8 s	14,878, 4.8 s	2,369, 2.9 s
S2c, subagent 2 in the second round	17,242, 5.4 s	2,369, 2.9 s	2,369, 2.9 s
all ten requests	116,147 read, 49.9 s	86,332 read, 44.6 s	73,892 read, 42.9 s
evicted	3	1	2

With both, every return resumes its conversation (2.8 to 3.0 s). The second eviction is S1 at P4, by 5 MB: three conversations of 0.8 to 1.1 GiB and the one being switched in fill 2816 MiB.

Why it builds on #1163: borrowing keeps one more conversation in the cache, and with the reserve unbounded that pushed the main agent out - this PR alone on v0.1.40.1, same run: P3 read 34,728 tokens again (12.1 s instead of 2.9 s), all ten requests 54.5 s instead of 42.9 s.

The answers: with the fixed placement all ten equal #1163 alone (S2 borrowed gives the same tokens as S2 moved in the old way), and the main agent's four answers equal a run without subagents.

Tests: conversation_cache_test (which matches borrow and which do not, pinning against slot and byte eviction and the drop of superseded copies, the refusal when the outgoing conversation fits only in the donor's room) and conversation_file_test on Windows; conversation_snapshot_test (on the GPU: the borrowed prefix and its checkpoint give the same bytes as restoring the whole conversation and rewinding to the checkpoint, and the parked image stays unchanged) on Windows with CUDA. The same run without --mtp borrows the same way.

[midhatn]

Tested a local port of this + #1163 onto v0.1.40.2 on Windows, Ryzen 9 7940HS / RTX 4070 Laptop 8 GB / 64 GB RAM, IQ3_S, 64K context. Both builds used the same toolchain, 36 GiB resident expert budget, two conversation slots and a 2 GiB cache.

Ten turns switched between a parent and two workers with deliberately different inventory answers (422 vs 1422). Every answer was correct:

Across the ten turns	Unchanged source build	#1163 + #1164 port
Newly read prompt tokens	29,176	21,323
Prompt processing	103.77 s	79.09 s
Total request wall time	137.28 s	116.49 s
Generated tokens	675	731

The final return to worker B read 8,082 tokens on the control versus 28 on the patched build. This is one complete sequence per build, with sampled high-reasoning output; the generated histories differ, so the 15% wall-time reduction is workload evidence rather than a deterministic speed guarantee. Separate pinned-document queries also passed.

One integration detail for the rebase: v0.1.40.2 now has permanently pinned prefixes. I preserved both entry.pinned() and the borrowed donor's serial in victim selection. A new three-entry test (pinned document A, borrowed donor B, ordinary C, then insert D) confirms only C is evicted. All six conversation tests passed on MSVC/CUDA. This looks useful for single-model agent switching.

Follow-up with a larger shared system prefix: the earlier 100-record fixture did not reach the default 2,048-token root threshold, so it tested storage/eviction behavior rather than borrowing. I repeated the ten-turn schedule with a 2,833-token common root and checked the engine's actual borrowed event.

Both arms reused that root on B's first turn. On returning to A and B later, the control read 6,203 and 6,523 tokens, while the #1163 + #1164 port read only 26 and 27. All distinct-answer checks passed. Across the sequence, newly read tokens fell from 33,731 to 21,058, and prompt processing from 116.85 to 75.36 s. The candidate logged the borrowed restore; the control did not.

This is still one sampled sequence per arm and a combined port, not an isolated attribution to #1164. Output totals differed (1,235 vs 725), so the 174.05 vs 108.67 s total wall times also include generation differences. The specific improvement supported by the cache trace is avoiding the two history rereads after borrowing the common root.

[BlueKingMuch]

Thanks @midhatn for porting and testing this, and for repeating it with a root above the 2,048-token threshold: the returns to A and B (26 and 27 tokens read instead of 6,203 and 6,523) are the case #1164 is for.

I've updated this PR for 0.1.40.3: a merge of v0.1.40.3 and one commit on top. Your integration point is what the merge does too: parking the outgoing conversation evicts neither a conversation that holds a pinned prefix (pin=N, since 0.1.40.2) nor the conversation a borrow comes from. The commit adds two details:

• A match that reaches the pinned prefix takes the conversation instead of borrowing from it. A borrow copies the donor's checkpoints up to the match, so the borrower would carry the pin as well; once parked, a second conversation would hold it and never be evicted.
• The refusal before evicting anything counts the conversations that hold a pinned prefix along with the donor. When they fill the slots or the budget between them, the outgoing conversation would otherwise find nothing to evict and not be parked; now the donor is taken whole as before.

The borrow's guard is now guard()/unguard() instead of pin()/unpin(), since "pinned" means pin=N.

Measured on 0.1.40.3, this against #1163 on 0.1.40.3, with the setup from the description (RTX 4080 SUPER 32 GB, Windows 11, IQ3_S, 262K context, --conversation-cache-mib 2816 --conversation-cache-slots 3, --adapt-every 1000000 --expert-cache 8031, greedy; P1 S1 P2 S2 S1b S2b P3 S1c S2c P4). Prompt tokens read, time to first token:

	#1163	#1163 + this PR
S2, new subagent while P is in the session	moves S1 in, S1 is overwritten	borrows 2,486 tokens from S1 (29 ms), S1 stays parked
S1b, subagent 1 goes on	14,878, 4.9 s	2,369, 2.9 s
all ten requests	86,332 read, 45.1 s	73,892 read, 43.2 s
evicted	1	2

With this PR every return resumes its conversation (2.8 to 3.0 s), and all ten answers are the same in both runs.

With a pinned prefix (a 20,014-token document sent with pin=N, five questions on it, and between them four subagents with a shared 6,018-token system prompt), each request reads as much as on #1163 alone and gives the same answer. The subagents borrow the system prompt from their parked sibling, where #1163 alone takes the sibling whole; the questions never borrow, they resume at the pinned prefix (29 tokens read, 0.4 s). At the end three conversations are parked instead of two, after two evictions (none on #1163 alone); the document was never among them (every later question still resumed at it).

Tests on Windows with CUDA: conversation_cache_test (4,744 checks, now with your case of a pinned prefix, the donor and an ordinary conversation, only the ordinary one evicted; the refusal with both in every slot; matches on and past a pinned prefix), conversation_snapshot_test (4,041), conversation_file_test (169).

[midhatn]

Thanks for incorporating the pinned-document/donor/ordinary-entry case. I reviewed cf7f3e8: keeping guard() separate from the persistent pin, and taking a match that reaches the pin rather than copying that pin into another parked conversation, address two distinct eviction problems.

For clarity, my measurements above are from the earlier local port. I have not rerun your new head, so they should not be counted as validation of the added pin-duplication and full-slot refusal cases. The source and test additions look aligned with those cases; no additional change requested from this review.

[edt-xx]

I run an agentic workload with prompts from about 70K to 105K. With #1164 + #1163 (and stats from #1114) installed I am seeing

Last request 84,024 of 84,075 prompt tokens reused
Reused since start 28,115,329 tokens (92% of all prompt tokens)

without this the reuse percentage is in the low 80s.

[Niko1221]

Thank you, this is in for 0.1.45, co-authored with you. It needed a hand rebase because 0.1.42 added pinned shared prefixes with their own eviction rule. On our RTX 3060 (IQ3_XXS) the sibling pattern A, B1, A, B2, A went from 13.35 s to 0.80 s on A's last turn (5 of 5 pairs), same greedy output; the sibling's own turn costs about 0.2 s more.

[Niko1221]

Thanks for the pull request.

Your change is already in main for 0.2.0-beta, with your credit, so there is nothing left for you to do here. We're closing it because we're resetting the tracker for 0.2.0-beta: there are more than 1,300 open issues and pull requests, and we're closing everything that isn't being worked on right now.

If part of your work did not make it in and you want it considered, open a new pull request on top of main that follows CONTRIBUTING.md, or comment /reopen after fixing the title area prefix and the template.