Repository navigation
qwen35: geist_session_pin_prefix + reset leaves an inconsistent DeltaNet state (silent wrong logits) #547
Description
geist_session_pin_prefix is tagged STABLE ("subsequent geist_session_reset() truncates the cache back to this prefix length"), but on the qwen35 hybrid the state after reset() is wrong: the attention KV prefix survives while the DeltaNet recurrent state is zeroed, so the next prefill continues from an inconsistent state. There is no guard; the call returns GEIST_OK.
Repro (pinned 7a0b421, qwen3.5-0.8b-q8_0.gguf, backend cpu_neon, M1 Max):
// A: fresh session, prefill_tokens(prefix + query) -> reference logits
// B: pin_prefix(prefix); prefill_tokens(query) -> logits
// C: (session B) reset(); prefill_tokens(other); reset(); prefill_tokens(query) -> logitsbackend=cpu_neon arch=qwen35
tokens prefix=20 query=9 vocab=248320
B vs A max|dlogit|=0.0000 argmax 271 vs 271
C vs A max|dlogit|=6.7348 argmax 198 vs 271
Pinning itself is fine (B == A); every turn after the first reset diverges (C). Reading the code, reset keeps the KV cache up to prefix_length but zeroes the DeltaNet state (src/archs/transformer/forward/step.c ~387–424), and pin_prefix (arch_ops.c:753) does not checkpoint the DeltaNet state of the pinned prefix.
Expected: either restore the DeltaNet state of the pinned prefix on reset (the speculative-decode path already has deltanet_txn_* checkpoints), or return GEIST_E_UNSUPPORTED from pin_prefix on hybrid architectures until that exists, so callers fall back to re-prefilling. Qwen3 (attention-only) is not affected.
Full test program (links only libgeist.a):
pin.c
#include "geist.h"
#include "geist_util.h"
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
static float *copy_logits(struct geist_session *s, size_t *n) {
const float *l = geist_session_peek_logits(n, s);
if (!l) { fprintf(stderr, "no logits\n"); exit(3); }
float *c = malloc(*n * sizeof *c); memcpy(c, l, *n * sizeof *c); return c;
}
static double diff(const float *a, const float *b, size_t n) {
double m = 0; for (size_t i = 0; i < n; i++) m = fmax(m, fabs(a[i] - b[i])); return m;
}
static size_t argmax(const float *a, size_t n) { size_t k = 0; for (size_t i = 1; i < n; i++) if (a[i] > a[k]) k = i; return k; }
int main(int argc, char **argv) {
struct geist_backend *be; struct geist_model *m;
if (geist_backend_create("auto", nullptr, nullptr, &be) || geist_model_load(argv[1], be, &m)) return 1;
printf("backend=%s arch=%s\n", geist_backend_name(be), geist_model_arch(m));
struct geist_session_opts o = {0};
struct geist_session *a, *b;
if (geist_session_create(m, be, &o, &a) || geist_session_create(m, be, &o, &b)) return 2;
const char *prefix = "Du bist EVA, eine Androidin an Bord der Helios. Antworte knapp und freundlich. ";
const char *query = "Alex fragt: Wo ist die Krankenstation?", *other = "Alex sagt: Ich habe Hunger und bin müde.";
geist_token_t P[512], Q[256], R[256], PQ[768]; size_t np, nq, nr;
if (geist_session_tokenize(a, prefix, 512, P, &np) || geist_session_tokenize(a, query, 256, Q, &nq) ||
geist_session_tokenize(a, other, 256, R, &nr)) return 4;
memcpy(PQ, P, np * sizeof *P); memcpy(PQ + np, Q, nq * sizeof *Q);
size_t n;
if (geist_session_prefill_tokens(a, np + nq, PQ)) return 5;
float *ref = copy_logits(a, &n); /* A: plain prefix+query */
if (geist_session_pin_prefix(b, np, P)) { printf("pin_prefix: unsupported/failed\n"); return 0; }
if (geist_session_prefill_tokens(b, nq, Q)) return 6;
float *pinned = copy_logits(b, &n); /* B: pinned, then query */
if (geist_session_reset(b) || geist_session_prefill_tokens(b, nr, R)) return 7;
if (geist_session_reset(b) || geist_session_prefill_tokens(b, nq, Q)) return 8;
float *again = copy_logits(b, &n); /* C: reset, other, reset, query */
printf("tokens prefix=%zu query=%zu vocab=%zu\n", np, nq, n);
printf("B vs A max|dlogit|=%.4f argmax %zu vs %zu\n", diff(pinned, ref, n), argmax(pinned, n), argmax(ref, n));
printf("C vs A max|dlogit|=%.4f argmax %zu vs %zu\n", diff(again, ref, n), argmax(again, n), argmax(ref, n));
return 0;
}
Activity
geisten commentedon Oct 1, 2026
Fixed on main by de37141 (PR #512, originally #452):
geist_session_pin_prefixnow returnsGEIST_E_UNSUPPORTEDwhen the model has DeltaNet layers andn > 0(src/archs/transformer/arch_ops.c:765-771), so a reset can no longer leave the recurrence out of sync with the KV prefix. The report was against 7a0b421, which predates that commit.Actually restoring the recurrent state to a prefix is the snapshot/restore feature tracked in #548.
Generated by Claude Code