You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
API: session snapshot/restore (or fork) for all architectures, incl. qwen35 hybrid #548
(or an in-memory geist_session_fork that copies the state into a second session).
Why: game and agent runtimes re-run many short requests against one long, mostly constant context (character sheet, situation, recent turns). On a 0.8B qwen35 model, 90–95 % of each decision is prefill of ~700 tokens (≈2.5 s on M1 Max, ≈16 s on Zen 5 with Q8_0, see #504). With a snapshot after the constant part, each request would only prefill its ~100 new tokens. It would also allow scoring several candidate continuations from the same state (log-prob of a whole label instead of only its first token), which today needs pin_prefix + reset and therefore only works on attention-only models.
pin_prefix covers part of this for attention-only models but is wrong on the qwen35 hybrid (separate bug report). The internals seem to be there already: the speculative-decode path checkpoints and rolls back the DeltaNet state (deltanet_txn_*, kv_truncate, src/archs/transformer/arch_ops.c ~141–221). The state of a 0.8B qwen35 session is a few MB, so a copy is cheap compared to a prefill.
A minimal version that only supports restore into the same session/model (no on-disk format, no cross-version stability) would already cover the use case.
Small update: the "pin_prefix is wrong on the qwen35 hybrid" bug (#547) is closed. Since de37141 (#512 / #452), geist_session_pin_prefix returns GEIST_E_UNSUPPORTED on DeltaNet models, so this feature request is now the only route to a reusable prefix there. The deltanet_txn_* code referenced above is now at arch_ops.c ~98-230.
Activity
geisten commentedon Oct 1, 2026
Bug report referenced above: #547.
geisten commentedon Oct 1, 2026
Small update: the "pin_prefix is wrong on the qwen35 hybrid" bug (#547) is closed. Since de37141 (#512 / #452),
geist_session_pin_prefixreturnsGEIST_E_UNSUPPORTEDon DeltaNet models, so this feature request is now the only route to a reusable prefix there. Thedeltanet_txn_*code referenced above is now atarch_ops.c~98-230.Generated by Claude Code
Merge pull request #603 from geisten/claude/project-thread-xm9brj