Repository navigation

serve: commit only the tokens a verify window hands out - #652

Closed

anon761 wants to merge 629 commits into

Niko1221:mainfrom

anon761:serve-commit-emitted

Closed

serve: commit only the tokens a verify window hands out#652
anon761 wants to merge 629 commits into
Niko1221:mainfrom
anon761:serve-commit-emitted

This pull request is big! We're only showing the most recent 250 commits

Commits on Sep 30, 2026
Commits on Oct 1, 2026
Commits on Oct 2, 2026
- README, MODELS: the AMD baseline - RX 9070 XT + Ryzen 9 3900X + 47 GB RAM (Q2_0, IQ2_XS, the Coder)
  
  Niko1221
  and
  claude
  committed
  1678de3
- serve: "messages" sent as a JSON string is decoded; a non-object entry is a 400 (#460)
  
  Niko1221
  and
  claude
  committed
  1281f0f
- Windows AMD: the HIP runtime next to strata.exe, not an AMD driver's System32 copy (#468, #461)
  
  Niko1221
  and
  claude
  committed
  b60c02c
- engine: the low-RAM resident mode engages on Windows 32 GB PCs (#467)
  
  Niko1221
  and
  claude
  committed
  98fa6fc
- setup: an earlier config that does not parse is skipped, not a crash; configs are written atomically (#459)
  
  Niko1221
  and
  claude
  committed
  fa16567
- setup (--backend hip): a system ROCm without the HIP development files falls back to the wheels (#446)
  
  Niko1221
  and
  claude
  committed
  749d957
- calibrate: the engine's arguments come from the server's engine_args; a failed tuning says why (#447)
  
  Niko1221
  and
  claude
  committed
  5a30024
- serve, setup: name the small split card that caps the prompt chunk; setup recommends the big card alone (#448)
  
  Niko1221
  and
  claude
  committed
  9f60d13
- /metrics: each request's speculative draft counts, and their sums in the totals (#457)
  
  Niko1221
  and
  claude
  committed
  eb8da1d
- setup (--gguf-dir): GGUFs Strata cannot run are named as such, with the --family/--model to use (#444)
  
  Niko1221
  and
  claude
  committed
  6a09387
- docs: the Coder is weaker outside code, Chinese/CJK text included; the unpruned sizes for that (#438)
  
  Niko1221
  and
  claude
  committed
  d6a91e2
- Merge branch 'f-135-eng' into rel-0.1.35
  Niko1221
  committed
  d8d4ad3
- Merge branch 'f-135-small' into rel-0.1.35
  Niko1221
  committed
  7b600d5
- Engine 0.1.35: Windows AMD uses its bundled HIP runtime (#468 #461), the low-RAM resident mode on Windows 32 GB PCs (#467), fixes #460 #459 #446 #447 #457 #448 #444; setup requires it
  
  Niko1221
  and
  claude
  committed
  d9ab843
- A prompt cancelled mid-read is logged and recorded with the tokens it read (#471)
  
  Niko1221
  and
  claude
  committed
  e3bdd74
- "The draft head does not fit" says what to change (#474)
  
  Niko1221
  and
  claude
  committed
  f2aaba1
- UPDATE.bat / update.sh: update Strata without starting the model (#475)
  
  Niko1221
  and
  claude
  committed
  64987c8
- Opt-in: keep what the expert cache learned across restarts (--expert-profile-save, #477)
  
  Niko1221
  and
  claude
  committed
  6d0e648
- Engine 0.1.36: a cancelled prompt logged as read so far (#471), the draft-head hint (#474), UPDATE.bat (#475), --expert-profile-save (#477); setup requires it
  
  Niko1221
  and
  claude
  committed
  5ccf3a7
- #136: fused int8 tensor-core experts for the Q2_0 pack's prompt path, grouped on the GPU (opt-in STRATA_PF_FUSED=1)
  
  Niko1221
  and
  claude
  committed
  b750bbc
- #136: the fused experts take a 512-slot streamed ring (a whole layer's experts) by default
  
  Niko1221
  and
  claude
  committed
  6a0d56c
- #136 P3: with the fused experts the prompt path's MoE buffers hold only what the fused kernels need
  
  Niko1221
  and
  claude
  committed
  6172e4a
- #136: the fused experts take a 1024-slot streamed ring by default (two layers' experts)
  
  Niko1221
  and
  claude
  committed
  c2bf349
- Decode on thread-block clusters (sm_90+): the QSA top-k and the greedy argmax, the same ids and tokens
  
  Niko1221
  and
  claude
  committed
  bbe3d2a
- #136: the fused int8 prompt experts for the native IQ packs (gate/up IQ2_XXS IQ2_XS IQ2_S IQ3_XXS IQ3_S IQ4_XS, down Q2_0 IQ4_NL; opt-in STRATA_PF_FUSED=1)
  
  Niko1221
  and
  claude
  committed
  3e5880d
- Merge branch 'f-136-fused' into rel-0.1.36
  Niko1221
  committed
  bb47c5c
- Merge the 0.1.36 speed work: fused Q2_0 prompt experts on by default (STRATA_PF_FUSED=0 off), the native IQ packs' fused kernels opt-in (=1), cluster decode kernels; the fused test waits for its up…
  
  Niko1221
  and
  claude
  committed
  b3e9545
- docs: 0.1.36's speed (fused Q2_0 prompt experts, cluster decode kernels) and their switches
  
  Niko1221
  and
  claude
  committed
  78f071c
- docs: the cluster decode kernels run on sm_90+ (RTX 50), not RTX 40
  
  Niko1221
  and
  claude
  committed
  36fa455
- HIP: time the PCIe probe on the host on Windows
  
  authored and
  Niko1221
  committed
  da719fc
- HIP: count the desktop's VRAM on Windows (WDDM budget, STRATA_WDDM_BUDGET)
  
  authored and
  Niko1221
  committed
  a596f15
- serve: end an engine that lost step with the server instead of waiting forever (#481)
  
  Niko1221
  and
  claude
  committed
  e07540f
- serve: the engine-exited-before-READY error shows this start's last 20 log lines (#496)
  
  Niko1221
  and
  claude
  committed
  f7df4ef
- serve: the start narrator says what happens to the experts with --mmap-experts (#505)
  
  Niko1221
  and
  claude
  committed
  6e05aa9
- setup: honour HF_ENDPOINT, and say where the model files go (#495)
  
  Niko1221
  and
  claude
  committed
  aa0e21b
- setup: UD-Q4_K_XL across GPUs without the RAM budget when the RAM holds it; no budget on a --gpus start (#498)
  
  Niko1221
  and
  claude
  committed
  869ebad
- setup: --vram-reserve-mib N, and INSTALL.md documents it (#493)
  
  Niko1221
  and
  claude
  committed
  b9c2d5e
- PCIe probe (#485): best of four timed bursts, a continuous share, the samples logged
  
  Niko1221
  and
  claude
  committed
  4e308eb
- docs: Windows AMD's free-VRAM figure within the WDDM budget (#380) and the host-timed PCIe probe (#377)
  
  Niko1221
  and
  claude
  committed
  100ebe4
- Merge branch 'f-137-srv' into rel-0.1.37
  Niko1221
  committed
  97fc539
- Engine 0.1.37: a silent engine is restarted (#481), Windows AMD counts the desktop's VRAM (#380 #377 #497), a steadier PCIe probe (#485), fixes #496 #495 #498 #505 #493; setup requires it
  
  Niko1221
  and
  claude
  committed
  db4f91a
- Prompt path: an MMQ group's experts gathered in one launch, after one wait, released by one event
  
  authored and
  Niko1221
  committed
  d541220
- Prompt path: the first chunk's n-gram (PLE) rows read beside layer 0, 256 at a time
  
  authored and
  Niko1221
  committed
  6639c7b
- ple_land: the rows' upload, its event and the wait before a refill are checked
  
  authored and
  Niko1221
  committed
  4f7b3e8
- Prompt path: a stager buffer's first job of a generation waits for the previous DMA from it
  
  authored and
  Niko1221
  committed
  3a91b9c
- prefill: the draft layer's batched K/V for a ring too (KV streaming), and STRATA_DRAFT_TIMING
  
  authored and
  Niko1221
  committed
  ba790d6
- qsa_prompt_attn: Q4_0 K and V on tensor cores (mode 4) - long prompts with --kv q4_0 no longer take the old kernel
  
  authored and
  Niko1221
  committed
  778e1f6
- qsa_prompt_attn: Q4_0 mode 4 on sm_80+ only (#452 follow-up)
  
  Niko1221
  and
  claude
  committed
  79670a6
- prefill: DeltaNet recurrence with the three value heads of a key head in one thread (1.44x), output norm without its dead FP32 store (1.5x); same bits
  
  authored and
  Niko1221
  committed
  a7ce31b
- native experts: Q5_0 GPU kernel (#222)
  
  Rhonstin
  authored and
  Niko1221
  committed
  1583054
- PLE: lift the stale Q5_0 + --ple-io direct refusal (#296)
  
  Rhonstin
  authored and
  Niko1221
  committed
  7e0da6c
- prefill: use active QSA top-k bounds on Turing with large KV capacity
  
  authored and
  Niko1221
  committed
  474a94b
- qsa_topk_active_parity checks the active-bound dispatch on any CUDA card (#512 follow-up)
  
  Niko1221
  and
  claude
  committed
  48b7d6f
- experts: the verify window's PCIe call launches for the groups it has; SwiGLU and q8_1 in one pass (bitwise)
  
  authored and
  Niko1221
  committed
  0bc1594
- decode: wait for an adaptive expert swap before reading the residency table
  
  constantindjonkam
  authored and
  Niko1221
  committed
  322ea76
- fix: --dump-logits must not promise rows it will never write
  
  constantindjonkam
  authored and
  Niko1221
  committed
  69de404
- decode: #463 cleanup - one wait in apply_pending, the serve call site indented, shorter comments
  
  Niko1221
  and
  claude
  committed
  1e5f6c1
- IQ4_XS expert rows on the AVX-2 path: the multi-token kernel this format never had
  
  pipeob0
  authored and
  Niko1221
  committed
  e7b415c
- IQ4_XS's AVX-2 kernel on AVX-2 CPUs only; native_expert_parity skips the AVX-512 check it has no kernel for (#415 follow-up)
  
  Niko1221
  and
  claude
  committed
  5696360
- Adaptive tier: --adapt-decay
  
  authored and
  Niko1221
  committed
  c38dc71
- Start: read the expert arena unbuffered when the file cache cannot help (Windows; part 1 of #285)
  
  authored and
  Niko1221
  committed
  d82d805
- load_experts_gguf: the header declares the new parameter
  
  authored and
  Niko1221
  committed
  efb552e
- File tier: read the GGUF in place unbuffered when the file cache cannot keep it beside the RAM budget (Windows; #286 ported into 0.1.31's tier)
  
  authored and
  Niko1221
  committed
  25416f3
- File tier: a pack's experts.bin is read unbuffered too (Windows)
  
  authored and
  Niko1221
  committed
  536392f
- pinned: honor STRATA_NO_LARGEPAGES on Linux; name the hugetlb pool shortfall
  
  yannickloth
  authored and
  Niko1221
  committed
  07b4a55
- tools/hip: gfx1201 hipBLASLt table for hipBLASLt 1.2.2 (ROCm 7.2.4)
  
  authored and
  Niko1221
  committed
  8fa04d1
- AMD: ship the gfx1200 hipBLASLt table, with the with/without measurements
  
  authored and
  Niko1221
  committed
  0cbc9ca
- HIP: run the MTP prompt pass per group by default
  
  authored and
  Niko1221
  committed
  79ac8ad
- multi-GPU: a second GPU as an opt-in expert-cache tier (--peer-device N)
  
  q8atnight
  authored and
  Niko1221
  committed
  9d9a251
- peer tier: refuse --peer-device with --layer-split or --expert-cache-device1..3 (both would use the same second card); --help lines for the --peer-* flags
  
  q8atnight
  authored and
  Niko1221
  committed
  fe95184
- docs: the refused --peer-device combinations
  
  q8atnight
  authored and
  Niko1221
  committed
  7854974
- prefill: with --peer-device the fused experts stay off even when the peer declines the prompt path (#531 follow-up)
  
  Niko1221
  and
  claude
  committed
  734273a
- kernels: opt-in small-window GR down projection (STRATA_GR_DOWN_MAX4)
  
  CC-David-CC
  authored and
  Niko1221
  committed
  4c3f599
- fused_gr: STRATA_GR_DOWN_MAX4 read once (#443 follow-up)
  
  Niko1221
  and
  claude
  committed
  92883e8
- A 6 GB card starts with a small expert cache, or says what is short (#496)
  
  Niko1221
  and
  claude
  committed
  e1ee248
- hip_compat: the peer-access calls (#531 follow-up)
  
  Niko1221
  and
  claude
  committed
  0d08349
- #496: the engine chooses a small card's reserve; setup only gives the tip
  
  Niko1221
  and
  claude
  committed
  4731a9b
- Engine 0.1.38: faster prompts (#372 #374 #413 #452), Q5_0 experts on the GPU (#473), IQ4_XS on AVX-2 (#415), unbuffered loads on Windows (#357 #362), --peer-device (#531), a 6 GB card starts (#496)…
  
  Niko1221
  and
  claude
  committed
  5123c97
- prefill: the resident lookup in the streamed walk indexes by the model's expert count (#531 follow-up)
  
  Niko1221
  and
  claude
  committed
  421a31d
- prefill: the fused layout's smaller MoE buffers only when the native kernels cover EVERY layer
  
  Niko1221
  and
  claude
  committed
  220e0e8
- decode: STRATA_ADAPT_NOWAIT=1, an A/B switch for #463's wait
  
  Niko1221
  and
  claude
  committed
  b4b79b0
- setup --update: skip a strata-*.json that is no model config instead of KeyError: 'args' (#549)
  
  Niko1221
  and
  claude
  committed
  067bfda
- serve: the "exceeds the context" 400 says how to get past it (#545)
  
  Niko1221
  and
  claude
  committed
  dca932f
- #560 #516: an AMD card on a Linux desktop gets a recommended VRAM reserve (3072 MiB)
  
  Niko1221
  and
  claude
  committed
  b48aad2
- serve: log when a reply reaches max tokens still thinking, and what helps (#530)
  
  Niko1221
  and
  claude
  committed
  080bafa
- decode_cluster_parity: wait for every upload before the graph replays and timings use it (#548, as #536)
  
  Niko1221
  and
  claude
  committed
  993e17f
- #542: MMQ's device facts from cudaDeviceGetAttribute; warn when libcudart is older than the headers
  
  Niko1221
  and
  claude
  committed
  63e8b50
- docs: an OpenCode config example (#543)
  
  Niko1221
  and
  claude
  committed
  ef30aab
- setup: a start prints the settings it uses (#564)
  
  Niko1221
  and
  claude
  committed
  ae6dd61
Commits on Oct 3, 2026
- #486: a weight arena that does not fit says how much VRAM was free
  
  Niko1221
  and
  claude
  committed
  bd649ac
- serve: check the Host header on every request (DNS rebinding)
  
  Niko1221
  and
  claude
  committed
  e1a0f97
- serve: refuse cross-site browser requests without an API key
  
  Niko1221
  and
  claude
  committed
  9d9b92f
- docs: the Coder is a 32 GB fit, not the pick for general use
  
  Niko1221
  and
  claude
  committed
  c2d6d33
- serve: no Host check when an API key is set
  
  Niko1221
  and
  claude
  committed
  7df6cbc
- Merge branch 'f-138-fixA' into f-138-prs
  Niko1221
  committed
  b20cdf5
- Merge branch 'f-138-sec' into f-138-prs
  Niko1221
  committed
  e4e0cee
- Merge branch 'f-138-fixB' into f-138-prs
  Niko1221
  committed
  99f3dbd
- serve: commit only the tokens a verify window hands out
  Anon
  committed
  cf14881

Commits

Commits on Sep 30, 2026

Commits on Oct 1, 2026