Repository navigation

cpu: IQ3_S / IQ2_S AVX2 decode without the per-index shift/mask/or (1.07-1.6x per row) - #930

Closed

sanastasiou wants to merge 846 commits into

Niko1221:mainfrom

sanastasiou:perf/avx2-iq3s-iq2s-decode

Closed

cpu: IQ3_S / IQ2_S AVX2 decode without the per-index shift/mask/or (1.07-1.6x per row)#930
sanastasiou wants to merge 846 commits into
Niko1221:mainfrom
sanastasiou:perf/avx2-iq3s-iq2s-decode

This pull request is big! We're only showing the most recent 250 commits

Commits on Oct 2, 2026
- setup: an earlier config that does not parse is skipped, not a crash; configs are written atomically (#459)
  
  Niko1221
  and
  claude
  committed
  fa16567
- setup (--backend hip): a system ROCm without the HIP development files falls back to the wheels (#446)
  
  Niko1221
  and
  claude
  committed
  749d957
- calibrate: the engine's arguments come from the server's engine_args; a failed tuning says why (#447)
  
  Niko1221
  and
  claude
  committed
  5a30024
- serve, setup: name the small split card that caps the prompt chunk; setup recommends the big card alone (#448)
  
  Niko1221
  and
  claude
  committed
  9f60d13
- /metrics: each request's speculative draft counts, and their sums in the totals (#457)
  
  Niko1221
  and
  claude
  committed
  eb8da1d
- setup (--gguf-dir): GGUFs Strata cannot run are named as such, with the --family/--model to use (#444)
  
  Niko1221
  and
  claude
  committed
  6a09387
- docs: the Coder is weaker outside code, Chinese/CJK text included; the unpruned sizes for that (#438)
  
  Niko1221
  and
  claude
  committed
  d6a91e2
- Merge branch 'f-135-eng' into rel-0.1.35
  Niko1221
  committed
  d8d4ad3
- Merge branch 'f-135-small' into rel-0.1.35
  Niko1221
  committed
  7b600d5
- Engine 0.1.35: Windows AMD uses its bundled HIP runtime (#468 #461), the low-RAM resident mode on Windows 32 GB PCs (#467), fixes #460 #459 #446 #447 #457 #448 #444; setup requires it
  
  Niko1221
  and
  claude
  committed
  d9ab843
- sycl: pipeline the expert cache's profile fill (model load 3-4x faster)
  
  maxfridbe
  and
  claude
  committed
  57bd713
- A prompt cancelled mid-read is logged and recorded with the tokens it read (#471)
  
  Niko1221
  and
  claude
  committed
  e3bdd74
- "The draft head does not fit" says what to change (#474)
  
  Niko1221
  and
  claude
  committed
  f2aaba1
- UPDATE.bat / update.sh: update Strata without starting the model (#475)
  
  Niko1221
  and
  claude
  committed
  64987c8
- Opt-in: keep what the expert cache learned across restarts (--expert-profile-save, #477)
  
  Niko1221
  and
  claude
  committed
  6d0e648
- Engine 0.1.36: a cancelled prompt logged as read so far (#471), the draft-head hint (#474), UPDATE.bat (#475), --expert-profile-save (#477); setup requires it
  
  Niko1221
  and
  claude
  committed
  5ccf3a7
- #136: fused int8 tensor-core experts for the Q2_0 pack's prompt path, grouped on the GPU (opt-in STRATA_PF_FUSED=1)
  
  Niko1221
  and
  claude
  committed
  b750bbc
- #136: the fused experts take a 512-slot streamed ring (a whole layer's experts) by default
  
  Niko1221
  and
  claude
  committed
  6a0d56c
- #136 P3: with the fused experts the prompt path's MoE buffers hold only what the fused kernels need
  
  Niko1221
  and
  claude
  committed
  6172e4a
- #136: the fused experts take a 1024-slot streamed ring by default (two layers' experts)
  
  Niko1221
  and
  claude
  committed
  c2bf349
- Decode on thread-block clusters (sm_90+): the QSA top-k and the greedy argmax, the same ids and tokens
  
  Niko1221
  and
  claude
  committed
  bbe3d2a
- #136: the fused int8 prompt experts for the native IQ packs (gate/up IQ2_XXS IQ2_XS IQ2_S IQ3_XXS IQ3_S IQ4_XS, down Q2_0 IQ4_NL; opt-in STRATA_PF_FUSED=1)
  
  Niko1221
  and
  claude
  committed
  3e5880d
- Merge upstream main (0.1.35) into b70
  
  maxfridbe
  and
  claude
  committed
  e5c2836
- Merge branch 'f-136-fused' into rel-0.1.36
  Niko1221
  committed
  bb47c5c
- Merge the 0.1.36 speed work: fused Q2_0 prompt experts on by default (STRATA_PF_FUSED=0 off), the native IQ packs' fused kernels opt-in (=1), cluster decode kernels; the fused test waits for its up…
  
  Niko1221
  and
  claude
  committed
  b3e9545
- sycl: port upstream 0.1.35 (re-migration of its changed engine files)
  
  maxfridbe
  and
  claude
  committed
  41ad902
- docs: 0.1.36's speed (fused Q2_0 prompt experts, cluster decode kernels) and their switches
  
  Niko1221
  and
  claude
  committed
  78f071c
- docs: the cluster decode kernels run on sm_90+ (RTX 50), not RTX 40
  
  Niko1221
  and
  claude
  committed
  36fa455
- HIP: time the PCIe probe on the host on Windows
  
  authored and
  Niko1221
  committed
  da719fc
- HIP: count the desktop's VRAM on Windows (WDDM budget, STRATA_WDDM_BUDGET)
  
  authored and
  Niko1221
  committed
  a596f15
- serve: end an engine that lost step with the server instead of waiting forever (#481)
  
  Niko1221
  and
  claude
  committed
  e07540f
- serve: the engine-exited-before-READY error shows this start's last 20 log lines (#496)
  
  Niko1221
  and
  claude
  committed
  f7df4ef
- serve: the start narrator says what happens to the experts with --mmap-experts (#505)
  
  Niko1221
  and
  claude
  committed
  6e05aa9
- setup: honour HF_ENDPOINT, and say where the model files go (#495)
  
  Niko1221
  and
  claude
  committed
  aa0e21b
- setup: UD-Q4_K_XL across GPUs without the RAM budget when the RAM holds it; no budget on a --gpus start (#498)
  
  Niko1221
  and
  claude
  committed
  869ebad
- setup: --vram-reserve-mib N, and INSTALL.md documents it (#493)
  
  Niko1221
  and
  claude
  committed
  b9c2d5e
- PCIe probe (#485): best of four timed bursts, a continuous share, the samples logged
  
  Niko1221
  and
  claude
  committed
  4e308eb
- docs: Windows AMD's free-VRAM figure within the WDDM budget (#380) and the host-timed PCIe probe (#377)
  
  Niko1221
  and
  claude
  committed
  100ebe4
- Merge branch 'f-137-srv' into rel-0.1.37
  Niko1221
  committed
  97fc539
- Engine 0.1.37: a silent engine is restarted (#481), Windows AMD counts the desktop's VRAM (#380 #377 #497), a steadier PCIe probe (#485), fixes #496 #495 #498 #505 #493; setup requires it
  
  Niko1221
  and
  claude
  committed
  db4f91a
- sycl: port open upstream PRs #385 #463 #453 #374 #363 #407 #413 onto 0.1.35
  
  maxfridbe
  and
  claude
  committed
  88c3ace
- sycl: OpenAI logprobs on /v1/chat/completions (engine LP lines + server_intel)
  
  maxfridbe
  and
  claude
  committed
  4ba35fd
- Prompt path: an MMQ group's experts gathered in one launch, after one wait, released by one event
  
  authored and
  Niko1221
  committed
  d541220
- Prompt path: the first chunk's n-gram (PLE) rows read beside layer 0, 256 at a time
  
  authored and
  Niko1221
  committed
  6639c7b
- ple_land: the rows' upload, its event and the wait before a refill are checked
  
  authored and
  Niko1221
  committed
  4f7b3e8
- Prompt path: a stager buffer's first job of a generation waits for the previous DMA from it
  
  authored and
  Niko1221
  committed
  3a91b9c
- prefill: the draft layer's batched K/V for a ring too (KV streaming), and STRATA_DRAFT_TIMING
  
  authored and
  Niko1221
  committed
  ba790d6
- qsa_prompt_attn: Q4_0 K and V on tensor cores (mode 4) - long prompts with --kv q4_0 no longer take the old kernel
  
  authored and
  Niko1221
  committed
  778e1f6
- qsa_prompt_attn: Q4_0 mode 4 on sm_80+ only (#452 follow-up)
  
  Niko1221
  and
  claude
  committed
  79670a6
- prefill: DeltaNet recurrence with the three value heads of a key head in one thread (1.44x), output norm without its dead FP32 store (1.5x); same bits
  
  authored and
  Niko1221
  committed
  a7ce31b
- native experts: Q5_0 GPU kernel (#222)
  
  Rhonstin
  authored and
  Niko1221
  committed
  1583054
- PLE: lift the stale Q5_0 + --ple-io direct refusal (#296)
  
  Rhonstin
  authored and
  Niko1221
  committed
  7e0da6c
- prefill: use active QSA top-k bounds on Turing with large KV capacity
  
  authored and
  Niko1221
  committed
  474a94b
- qsa_topk_active_parity checks the active-bound dispatch on any CUDA card (#512 follow-up)
  
  Niko1221
  and
  claude
  committed
  48b7d6f
- experts: the verify window's PCIe call launches for the groups it has; SwiGLU and q8_1 in one pass (bitwise)
  
  authored and
  Niko1221
  committed
  0bc1594
- decode: wait for an adaptive expert swap before reading the residency table
  
  constantindjonkam
  authored and
  Niko1221
  committed
  322ea76
- fix: --dump-logits must not promise rows it will never write
  
  constantindjonkam
  authored and
  Niko1221
  committed
  69de404
- decode: #463 cleanup - one wait in apply_pending, the serve call site indented, shorter comments
  
  Niko1221
  and
  claude
  committed
  1e5f6c1
- IQ4_XS expert rows on the AVX-2 path: the multi-token kernel this format never had
  
  pipeob0
  authored and
  Niko1221
  committed
  e7b415c
- IQ4_XS's AVX-2 kernel on AVX-2 CPUs only; native_expert_parity skips the AVX-512 check it has no kernel for (#415 follow-up)
  
  Niko1221
  and
  claude
  committed
  5696360
- Adaptive tier: --adapt-decay
  
  authored and
  Niko1221
  committed
  c38dc71
- Start: read the expert arena unbuffered when the file cache cannot help (Windows; part 1 of #285)
  
  authored and
  Niko1221
  committed
  d82d805
- load_experts_gguf: the header declares the new parameter
  
  authored and
  Niko1221
  committed
  efb552e
- File tier: read the GGUF in place unbuffered when the file cache cannot keep it beside the RAM budget (Windows; #286 ported into 0.1.31's tier)
  
  authored and
  Niko1221
  committed
  25416f3
- File tier: a pack's experts.bin is read unbuffered too (Windows)
  
  authored and
  Niko1221
  committed
  536392f
- pinned: honor STRATA_NO_LARGEPAGES on Linux; name the hugetlb pool shortfall
  
  yannickloth
  authored and
  Niko1221
  committed
  07b4a55
- tools/hip: gfx1201 hipBLASLt table for hipBLASLt 1.2.2 (ROCm 7.2.4)
  
  authored and
  Niko1221
  committed
  8fa04d1
- AMD: ship the gfx1200 hipBLASLt table, with the with/without measurements
  
  authored and
  Niko1221
  committed
  0cbc9ca
- HIP: run the MTP prompt pass per group by default
  
  authored and
  Niko1221
  committed
  79ac8ad
- multi-GPU: a second GPU as an opt-in expert-cache tier (--peer-device N)
  
  q8atnight
  authored and
  Niko1221
  committed
  9d9a251
- peer tier: refuse --peer-device with --layer-split or --expert-cache-device1..3 (both would use the same second card); --help lines for the --peer-* flags
  
  q8atnight
  authored and
  Niko1221
  committed
  fe95184
- docs: the refused --peer-device combinations
  
  q8atnight
  authored and
  Niko1221
  committed
  7854974
- prefill: with --peer-device the fused experts stay off even when the peer declines the prompt path (#531 follow-up)
  
  Niko1221
  and
  claude
  committed
  734273a
- kernels: opt-in small-window GR down projection (STRATA_GR_DOWN_MAX4)
  
  CC-David-CC
  authored and
  Niko1221
  committed
  4c3f599
- fused_gr: STRATA_GR_DOWN_MAX4 read once (#443 follow-up)
  
  Niko1221
  and
  claude
  committed
  92883e8
- A 6 GB card starts with a small expert cache, or says what is short (#496)
  
  Niko1221
  and
  claude
  committed
  e1ee248
- hip_compat: the peer-access calls (#531 follow-up)
  
  Niko1221
  and
  claude
  committed
  0d08349
- #496: the engine chooses a small card's reserve; setup only gives the tip
  
  Niko1221
  and
  claude
  committed
  4731a9b
- Engine 0.1.38: faster prompts (#372 #374 #413 #452), Q5_0 experts on the GPU (#473), IQ4_XS on AVX-2 (#415), unbuffered loads on Windows (#357 #362), --peer-device (#531), a 6 GB card starts (#496)…
  
  Niko1221
  and
  claude
  committed
  5123c97
- prefill: the resident lookup in the streamed walk indexes by the model's expert count (#531 follow-up)
  
  Niko1221
  and
  claude
  committed
  421a31d
- prefill: the fused layout's smaller MoE buffers only when the native kernels cover EVERY layer
  
  Niko1221
  and
  claude
  committed
  220e0e8
- decode: STRATA_ADAPT_NOWAIT=1, an A/B switch for #463's wait
  
  Niko1221
  and
  claude
  committed
  b4b79b0
- setup --update: skip a strata-*.json that is no model config instead of KeyError: 'args' (#549)
  
  Niko1221
  and
  claude
  committed
  067bfda
- serve: the "exceeds the context" 400 says how to get past it (#545)
  
  Niko1221
  and
  claude
  committed
  dca932f
- #560 #516: an AMD card on a Linux desktop gets a recommended VRAM reserve (3072 MiB)
  
  Niko1221
  and
  claude
  committed
  b48aad2
- serve: log when a reply reaches max tokens still thinking, and what helps (#530)
  
  Niko1221
  and
  claude
  committed
  080bafa
- decode_cluster_parity: wait for every upload before the graph replays and timings use it (#548, as #536)
  
  Niko1221
  and
  claude
  committed
  993e17f
- #542: MMQ's device facts from cudaDeviceGetAttribute; warn when libcudart is older than the headers
  
  Niko1221
  and
  claude
  committed
  63e8b50
- docs: an OpenCode config example (#543)
  
  Niko1221
  and
  claude
  committed
  ef30aab
- setup: a start prints the settings it uses (#564)
  
  Niko1221
  and
  claude
  committed
  ae6dd61
Commits on Oct 3, 2026
- #486: a weight arena that does not fit says how much VRAM was free
  
  Niko1221
  and
  claude
  committed
  bd649ac
- serve: check the Host header on every request (DNS rebinding)
  
  Niko1221
  and
  claude
  committed
  e1a0f97
- serve: refuse cross-site browser requests without an API key
  
  Niko1221
  and
  claude
  committed
  9d9b92f
- docs: the Coder is a 32 GB fit, not the pick for general use
  
  Niko1221
  and
  claude
  committed
  c2d6d33
- serve: no Host check when an API key is set
  
  Niko1221
  and
  claude
  committed
  7df6cbc
- Merge branch 'f-138-fixA' into f-138-prs
  Niko1221
  committed
  b20cdf5
- Merge branch 'f-138-sec' into f-138-prs
  Niko1221
  committed
  e4e0cee
- Merge branch 'f-138-fixB' into f-138-prs
  Niko1221
  committed
  99f3dbd
- Several conversations at once: batch slots, pipelined layer split, per-stage dense weights
  
  blange48
  and
  claude
  committed
  ba707d8
- server: a client that stops reading early keeps the engine in step
  
  blange48
  and
  claude
  committed
  c67052c
- Conversation parking with --layer-split
  
  crazyaimachine
  and
  blange48
  committed
  e11166b
- Tests and docs: parking with a layer split, tools/parking_test.py, a Testing section
  
  blange48
  and
  claude
  committed
  220ccbb
- Parking with a layer split: the draft ring once, every stage's K/V reused, checkpoints moved not copied
  
  blange48
  and
  claude
  committed
  f67bf51
- serve: restart a dead engine for real - wait for the old process, retry the start, keep the last known context
  
  JeanP00l
  authored and
  Niko1221
  committed
  2cb4bde
- serve: restart() keeps 0.1.38's close() (PR #637 follow-up)
  
  Niko1221
  and
  claude
  committed
  d86dcb7
- serve: a malformed "tools" value is a 400, not a dropped connection (#592)
  
  Niko1221
  and
  claude
  committed
  e50592b
- File tier: decide unbuffered reads from the expert bytes it reads, again once the RAM copy is built (#577)
  
  Niko1221
  and
  claude
  committed
  f4a2f0a
- serve: read the /load and /unload body before replying (#630)
  
  authored and
  Niko1221
  committed
  4f3973a
- Load the native head and the logits before the expert arena; name the free VRAM when they fail (#620)
  
  Niko1221
  and
  claude
  committed
  7c77eae
- setup: running it again keeps the run config's keys it does not write (#629)
  
  Niko1221
  and
  claude
  committed
  ca8125c
- layer split: with explicit split points each GPU loads only its own layers' dense weights
  
  JeanP00l
  authored and
  Niko1221
  committed
  e50f266
- layer split trim: CUDA0 keeps every layer's router with --mmap-experts (PR #639 follow-up)
  
  Niko1221
  and
  claude
  committed
  efec569
- STRATA_ARENA_MMAP=1: the expert arena as a read-only mapped file (Linux), for small-RAM machines
  
  JeanP00l
  authored and
  Niko1221
  committed
  e30dd85
- layer_split: check it before the start, accept a JSON list, say the format with an example (#644)
  
  Niko1221
  and
  claude
  committed
  3982ec7
- adaptive swaps: gather the pin spans only when they are pinned (PR #640 follow-up)
  
  Niko1221
  and
  claude
  committed
  247b5a1
- setup: --no-browser / "open_browser": false keeps the chat page from opening (#609, #631)
  
  Niko1221
  and
  claude
  committed
  fe49e22
- make_profile: --reorder so a trace can rank ahead of a complete base
  
  yannickloth
  authored and
  Niko1221
  committed
  e90fbb4
- Parking with a layer split: the draft ring with the last stage, on its GPU; guards; trim log
  
  blange48
  and
  claude
  committed
  537588f
- setup: --vision-tokens N, the most image tokens a picture becomes (#625)
  
  Niko1221
  and
  claude
  committed
  f12b99f
- setup: recommend P-cores + half the E-cores as CPU expert workers on hybrid CPUs (#642)
  
  committed
  0e9814a
- q8_1 activations: keep the block's fp16 scale and sum finite (#606)
  
  Niko1221
  and
  claude
  committed
  19de7f4
- serve: end a reply that repeats one token 256 times in a row (#606)
  
  Niko1221
  and
  claude
  committed
  3903aab
- setup: --draft-vocab fr - the English/code subset plus the tokens of French text (#597)
  
  Niko1221
  and
  claude
  committed
  0cc4e86
- setup: a calibration on Linux HIP names the AMD card and is reused for it (#566)
  
  Niko1221
  and
  claude
  committed
  0129f22
- serve: a literal <think> / </think> in a message is encoded as text (#537)
  
  Niko1221
  and
  claude
  committed
  e43d072
- --ple-io direct on a rotational disk: warn at start, setup keeps the table in RAM, the stall report counts disk waits (#605)
  
  Niko1221
  and
  claude
  committed
  71d41cd
- docs: AMD_HIP.md - a calibration run by hand on Linux HIP is kept for its card (#566)
  
  Niko1221
  and
  claude
  committed
  e1e3aa9
- The expert arena checks the host RAM before it is allocated; the RAM probe reads cgroup v1 too (#633)
  
  Niko1221
  and
  claude
  committed
  d533fe9
- setup: STRATA_NVCC picks the CUDA toolkit; CUDA_HOME is looked at; glibc 2.43 + CUDA 12.9 and gcc 15 documented (#601)
  
  Niko1221
  and
  claude
  committed
  bb7c91b
- layer split trim: on by default in the HIP builds, opt-in on CUDA (PR #639 follow-up)
  
  Niko1221
  and
  claude
  committed
  12c099b
- STRATA_ARENA_MMAP: write experts.bin only when the drive has room (PR #640 follow-up)
  
  Niko1221
  and
  claude
  committed
  0f8cbeb
- serve + engine: opt-in "effort_position": "end" keeps the cached conversation when the effort changes (#458)
  
  Niko1221
  and
  claude
  committed
  61479a8
- Windows Pascal/Volta source build: link the shared CUDA runtime only (#585)
  
  Niko1221
  and
  claude
  committed
  c129c4c
- docs: the layer split's per-card dense weights (PR #639), STRATA_ARENA_MMAP (PR #640), make_profile --reorder (#587)
  
  Niko1221
  and
  claude
  committed
  2bc2dc1
- Hit rate: say the PCIe share beside it (#588)
  
  Niko1221
  and
  claude
  committed
  553f37b
- Document the decode window profiler (STRATA_VERIFY_PROFILE + STRATA_DECODE_TIMING); --gpu-stages points to it on a native pack (#610)
  
  Niko1221
  and
  claude
  committed
  587bb07
- web: a Conversation cache card in the Monitor tab (#596)
  
  Niko1221
  and
  claude
  committed
  0890960
- SECURITY.md: how to report privately, and what the server exposes (#544)
  
  Niko1221
  and
  claude
  committed
  0e17d1b
- Docs: the five reading errors #604 found (the sixth, the n-gram table's unbuffered reads, is in #605's commit)
  
  Niko1221
  and
  claude
  committed
  312f116
- web: a Model settings view for documented run config keys (#564)
  
  Niko1221
  and
  claude
  committed
  4e8d036
- serve: the OpenAI Responses API, POST /v1/responses (#451)
  
  committed
  0ad8f70
- web: the Model settings card's Save is the primary button (#564)
  
  Niko1221
  and
  claude
  committed
  f0489c9
- setup: Unsloth's UD-IQ4_XS as an experimental choice (#621)
  
  Niko1221
  and
  claude
  committed
  e809f7f
- File tier: the #577 re-check's log line on Windows only (the unbuffered reads exist there alone)
  
  Niko1221
  and
  claude
  committed
  f17d0ea
- strata_mcp: UD-IQ4_XS in the built-in model table, as in setup.py (#621)
  
  Niko1221
  and
  claude
  committed
  4f691be
- docs, setup help: --draft-vocab fr and its measured acceptance (#597)
  
  Niko1221
  and
  claude
  committed
  c062690
- serve tests: the long fixture answer alternates two tokens (one repeated token is now ended at 256, #606)
  
  Niko1221
  and
  claude
  committed
  a481806
- Hot VRAM resize, opt-in: a segmented expert cache, VRAM <reserve>, POST /v1/vram (#533)
  
  Niko1221
  and
  claude
  committed
  d595a2b
- setup: recommend the hybrid worker count only with more E-cores than P-cores (#642)
  
  Niko1221
  and
  claude
  committed
  2b47285
- verify: opt-in handshake trace for the gfx1030 timeouts, HIP A/B switches (#649)
  
  Niko1221
  and
  claude
  committed
  d5469a9
- prefill: step diagnosis and a HIP host-DMA switch for the gfx1201 prompt stalls (#579 #613 #541)
  
  Niko1221
  and
  claude
  committed
  645a579
- tools: conversation_restore_speed.py, decode after a conversation-cache restore vs live (#528)
  
  Niko1221
  and
  claude
  committed
  1807086
- serve: a checkpoint that fails on the GPU sync says the GPU stopped responding (#613)
  
  Niko1221
  and
  claude
  committed
  138ce43
- Merge branch 'f-139-eng' into f-139-rel
  Niko1221
  committed
  bdd5f50
- Merge branch 'f-139-ux' into f-139-rel
  Niko1221
  committed
  4fc8876
- Merge branch 'f-139-resp' into f-139-rel
  Niko1221
  committed
  ece2e5f
- Merge branch 'f-139-prs' into f-139-rel
  Niko1221
  committed
  6491760
- Merge branch 'f-139-big' into f-139-rel
  Niko1221
  committed
  0d5387f
- layer split trim: opt-in on HIP too (STRATA_STAGE_TRIM=1, PR #639)
  
  Niko1221
  and
  claude
  committed
  13f70a3
- Engine 0.1.39: the #577 file-tier regression fixed, the OpenAI Responses API (#451), #606 #620 #458 #533, PR batch; setup requires it
  
  Niko1221
  and
  claude
  committed
  cff608c
- Layer split comment: KV streaming and images do run across a split
  
  blange48
  and
  claude
  committed
  bd400f7
- Merge upstream main (0.1.38) into b70
  
  maxfridbe
  and
  claude
  committed
  a3277e3
- sycl: port upstream 0.1.38
  
  maxfridbe
  and
  claude
  committed
  daf3b7e
- qsa_select: the top-k past the register kernel's reach on 1,024 threads with a histogram per warp (CUDA) - 9-12x at 262K-524K cells, a 243K-token prompt +26% on an RTX 3060
  
  authored and
  Niko1221
  committed
  cce52db
- Added gfx1031 experimental support
  
  matthewjt
  authored and
  Niko1221
  committed
  5bd5eec
- forgot one spot to add in setup.py
  
  matthewjt
  authored and
  Niko1221
  committed
  78bb6dc
- setup: gfx1031 (RX 6700 XT) wording and a test (PR #524 follow-up)
  
  Niko1221
  and
  claude
  committed
  357723a
- sm_75: run the prompt path's BF16 products on the FP16 tensor cores
  
  authored and
  Niko1221
  committed
  f2fb7c1
- BF16 tensor-core path: default on for all of compute capability 7.x (Volta too)
  
  authored and
  Niko1221
  committed
  3f81bce
- iq_avx2: build the IQ2 sign table without a variable shift
  
  demetree
  authored and
  Niko1221
  committed
  dd0ce1e
- setup: --backend sycl, the Intel Arc engine (experimental, built from source; sycl/setup_intel.py)
  
  Niko1221
  and
  claude
  committed
  ab8729e
- perf(cuda,decode): zero-doorbell verify graph, sub-warp expert packing, shared-mem staging, and batched PLE/MTP
  
  stuchapin909
  authored and
  Niko1221
  committed
  cfd3b72
- fix(cuda,decode): resolve sm_120 IQ3_S generic-pointer fault and restore 100% bit-exactness with main
  
  stuchapin909
  authored and
  Niko1221
  committed
  055122c
- iq_avx2: the IQ2 sign table at compile time (no static constructor in an AVX2 file)
  
  Niko1221
  and
  claude
  committed
  016ea2e
- native experts: gate the AVX2 multi-token kernels on cpu_avx2_ok() (#394)
  
  authored and
  Niko1221
  committed
  e82947f
- prefill gemm below sm_80: Volta on the FP16 path by default, Turing opt-in, Pascal through fp32 (#655 #540 #395)
  
  committed
  ff6f9f1
- qsa_prompt_attn: tensor-core kernel for Volta (sm_70, mma.m8n8k4)
  
  authored and
  Niko1221
  committed
  06a90a2
- docs: NVIDIA_V100.md and the measurements of the Volta prompt attention
  
  authored and
  Niko1221
  committed
  7d6482a
- bench: distribution check of the Volta prompt attention (teacher-forced KL, controls)
  
  authored and
  Niko1221
  committed
  f23ea57
- docs: point the BF16 note at #540 (the BF16-via-FP16 PR #593 was closed in its favor)
  
  authored and
  Niko1221
  committed
  b078be0
- native experts without AVX2: Q2_0 layers on ggml-cpu, every AVX2 kernel gated (#595)
  
  authored and
  Niko1221
  committed
  b30e815
- Volta prompt attention and the Pascal BF16 path: compiled into the experimental build only (PR #600, #395 follow-up)
  
  Niko1221
  and
  claude
  committed
  800ca98
- cmake: STRATA_ENABLE_SYCL, the experimental Intel Arc engine (sycl/, off by default)
  
  Niko1221
  and
  claude
  committed
  5398213
- perf(qsa): cut the attention kernel's shuffles and q shared-memory traffic, bit-exact
  
  sskver
  authored and
  Niko1221
  committed
  fd95405
- perf(volta): address review - keep BF16X2 remainder GEMMs on cuBLAS, keep Inf, share sq with spt, atomic cc cache
  
  sskver
  authored and
  Niko1221
  committed
  5fa9e7f
- Merge PR #559 (blange48, crazyaimachine): batch slots, pipelined layer split, parking with a split
  
  Niko1221
  and
  claude
  committed
  e3978eb
- QSA attention: PR #540's kernel below sm_75 in the experimental build only
  
  Niko1221
  and
  claude
  committed
  8fb3c91
- router lookahead: an AVX1 dot and a plain fallback for CPUs without AVX2 (Strata_Dirigo)
  
  authored and
  Niko1221
  committed
  1a3ee38
- sycl: the stream-all prompt walk by default when the VRAM holds >90% of the experts
  
  maxfridbe
  and
  claude
  committed
  96ccd2a
- Merge PR #423 (maxfridbe): Intel Arc SYCL port of the engine, experimental
  
  Niko1221
  and
  claude
  committed
  2f78b4d
- hip: AMD gfx906 (Instinct MI50 / MI60, Radeon VII; wave64) as an opt-in build, STRATA_HIP_GFX906=ON
  
  JeanP00l
  authored and
  Niko1221
  committed
  4d79aea
- docs: gfx906 (Instinct MI50 / MI60, Radeon VII) build, kernels and measurements in AMD_HIP.md
  
  JeanP00l
  authored and
  Niko1221
  committed
  8534ee1
- gfx906: K-quant/Q5_1/Q8_0 experts take the CUDA layout; gr_down K-split opt-in; gfx906-aware ctest
  
  JeanP00l
  authored and
  Niko1221
  committed
  5c544dd
- gfx906: route the verify window's tokens in one BF16 projection + one top-k (bitwise the per-token calls)
  
  JeanP00l
  authored and
  Niko1221
  committed
  ccc09c2
- docs: gfx906 verify-window routing in the kernel table
  
  JeanP00l
  authored and
  Niko1221
  committed
  f0b3d17
- fused_gr: the gfx906 fast norm/up launches only in the gfx906 build (CUDA and RDNA did not compile)
  
  JeanP00l
  authored and
  Niko1221
  committed
  ce1e9b2
- core: optimize remote expert helper decode
  
  zhedang
  authored and
  Niko1221
  committed
  4d20d25
- refill: improve multi-GPU processing
  
  hash13
  authored and
  Niko1221
  committed
  44bd7ad
- gfx906 build (PR #638 follow-up): mrope.hpp's HIP include only in the gfx906 build; tests.list dropped
  
  Niko1221
  and
  claude
  committed
  e0c175b
- hip: add experimental gfx1012 support and legacy BLAS compatibility
  
  CC-David-CC
  authored and
  Niko1221
  committed
  73fcc76
- hip: preserve Q2_0 signed zero on gfx1012 with legacy HIP
  
  CC-David-CC
  authored and
  Niko1221
  committed
  a62c1cd
- hip: retain native wave synchronization on HIP 7
  
  CC-David-CC
  authored and
  Niko1221
  committed
  912f535
- bench: community report, 2x AMD Instinct MI50 (gfx906), Coder IQ1_M, 4K to 128K prompt tokens
  
  JeanP00l
  authored and
  Niko1221
  committed
  fa8dd5c
- prefill: a layer split's idle card streams a share of a one-chunk prompt's experts
  
  authored and
  Niko1221
  committed
  fc60278
- pinned: back the Linux expert arena with transparent huge pages
  
  Anon
  authored and
  Niko1221
  committed
  6d51272
- iq: gather the AVX2 codebook from the grid table (STRATA_IQ256_GATHER)
  
  yannickloth
  authored and
  Niko1221
  committed
  dd24a55
- iq_avx2: STRATA_IQ256_GATHER=0 keeps the scalar grid assembly (parse the value)
  
  Niko1221
  and
  claude
  committed
  3a64d02
- older CPUs (experimental): STRATA_ISA_FLOOR=avx|none, and setup compiles it instead of stopping (#394 #595 #623)
  
  Niko1221
  and
  claude
  committed
  1f57034
- setup: older NVIDIA GPUs through a second, experimental CUDA 12 engine; --cuda 12|13 (#395 #627 #540 #600 #601)
  
  Niko1221
  and
  claude
  committed
  9576724
- router_dot_parity: the router lookahead's AVX2 and AVX1 dots against a double reference
  
  Niko1221
  and
  claude
  committed
  870ebe9
- docs: Older GPUs (experimental) - the support matrix, the CUDA 12 engine and how to opt in
  
  Niko1221
  and
  claude
  committed
  a43e080
- setup: the offer to share a model across two cards checks the model's own engine (CUDA 12 or 13)
  
  Niko1221
  and
  claude
  committed
  58eee0a
- sycl: port 0.1.39 (re-migration with dpct 2025.3, 3-way merge per docs/INTEL.md)
  
  Niko1221
  and
  claude
  committed
  5047172
- sycl: stage_room reads the free VRAM (a layer split's later GPUs had none)
  
  authored and
  Niko1221
  committed
  84876f7
- sycl: strata-sycl.sh passes ONEAPI_DEVICE_SELECTOR into the container
  
  Niko1221
  and
  claude
  committed
  7ba023e
- #646 on 0.1.39b: the layer split and the CPU path as before, the resident plan for 8-token windows
  
  Niko1221
  and
  claude
  committed
  deee447
- setup: --backend sycl tests; Windows points at Linux (setup_intel reads sysfs, which WSL has not)
  
  Niko1221
  and
  claude
  committed
  ca1a7c0
- #646: the single-matrix mmvq does not stage the IQ grids (10-20% slower on the RTX 5070 at 3-8 columns)
  
  Niko1221
  and
  claude
  committed
  f945515
- prefill gemm: STRATA_BF16_TC=2, a test mode that runs the FP16 path on any card (gemm_bf16_parity on sm_80+)
  
  Niko1221
  and
  claude
  committed
  031ff94
- docs/INTEL_ARC.md: the experimental Intel Arc engine - what you need, the build, what is tested and what is not
  
  Niko1221
  and
  claude
  committed
  da3e9a1
- gfx906 build on 0.1.39: the segmented expert cache (#533) is CUDA-only there too; pinned.cu's std::max beside windows.h
  
  Niko1221
  and
  claude
  committed
  552da41
- #465 engine: batch slots for one GPU - warn instead of refuse, windows over the active slots, slot conversation cache, prompts read beside decoding slots, BYIELD
  
  Niko1221
  and
  claude
  committed
  2fbbfa2
- #465 server: "parallel": N, slots in /metrics, one status per request, a long prompt gives way to a short one
  
  Niko1221
  and
  claude
  committed
  bebfca9
- #465 setup: --parallel N, and the count it recommends for the card
  
  Niko1221
  and
  claude
  committed
  4795f98
- cpu: preserve full thread affinity on Windows and Linux (#626)
  
  authored and
  Niko1221
  committed
  ec35cf0
- #465 tools: batch_interleave_test.py (a prompt read beside decoding slots, BYIELD, a next turn from its slot); batch_test.py on Windows
  
  Niko1221
  and
  claude
  committed
  9564c73
Commits on Oct 4, 2026
- docs: Older CPUs (experimental) - what setup does on a CPU without AVX2, what was measured
  
  Niko1221
  and
  claude
  committed
  ab86bb4
- #465 a request left alone in a slot goes back to the solo path; docs: BATCHING.md for one GPU, measured on the RTX 5070
  
  Niko1221
  and
  claude
  committed
  9556298
- #465 engine: INFO slot_cache=0|1 beside batch_slots (the server sends a lone slot request back to the solo path only when the slots keep their conversations)
  
  Niko1221
  and
  claude
  committed
  831a90f
- prefill: the streamed ring is a byte budget, so a native pack sets its own slot count
  
  authored and
  Niko1221
  committed
  895a77b
- #465 docs: the final build's RTX 5070 numbers; the turn-checkpoint exactness check
  
  Niko1221
  and
  claude
  committed
  cef263f
- #465 serve/test_parallel: the first request decodes long enough to overlap the others on a busy PC
  
  Niko1221
  and
  claude
  committed
  0b651f8
- #465 setup: recommend "parallel" only where the experts mostly fit in VRAM
  
  Niko1221
  and
  claude
  committed
  1a56d71
- #583 on 0.1.39b: the ring's byte budget and the loan's corrected count opt-in (STRATA_RING_BYTES=1)
  
  Niko1221
  and
  claude
  committed
  8c968a5
- serve (auto): the chunk and the ring are one budget, and the ring is the better buy
  
  authored and
  Niko1221
  committed
  a93c2ac
- docs: STRATA_RING_BYTES=1 (#583, opt-in) with its RTX 5070 numbers and quality check
  
  Niko1221
  and
  claude
  committed
  06586e3
- hip_compat: cudaFuncAttributePreferredSharedMemoryCarveout (#646's fused_gr carveout on HIP)
  
  Niko1221
  and
  claude
  committed
  f166564
- setup: --remote-expert-opt recommended for 2+ GPU configs (#578)
  
  Niko1221
  and
  claude
  committed
  63d8f90
- #583 default on (owner): STRATA_RING_BYTES=0 opts out; DETAILS.md with the measured quality
  
  Niko1221
  and
  claude
  committed
  5423a69
- setup: the CUDA 12 engine's driver floor per OS (Windows 527.41 -> 528, Linux 525); the wheel pins match the 12.9.1 toolkit
  
  Niko1221
  and
  claude
  committed
  bc0fc49
- #583: the byte-budget ring only where it gains; short prompts keep 0.1.39's ring
  
  Niko1221
  and
  claude
  committed
  5f19911
- Merge branch 'f-139b-cpu' into f-139-rel
  Niko1221
  committed
  c6275a7
- Merge branch 'f-139b-old' into f-139-rel
  Niko1221
  committed
  baed889
- Merge branch 'f-139b-arc' into f-139-rel
  Niko1221
  committed
  8e08550
- Merge branch 'f-139b-spd' into f-139-rel
  Niko1221
  committed
  cb97e6c
- Merge branch 'f-139b-par' into f-139-rel
  Niko1221
  committed
  1189218
- docs: README refined + six translations
  Niko1221
  committed
  8b36fa0
- setup: Unsloth's UD-IQ4_XS is a regular choice (no longer experimental); UD-Q4_K_XL stays experimental
  
  Niko1221
  and
  claude
  committed
  c2d1f19
- setup: a tip (not a change) for images on a 12 GB card: --vram-reserve-mib 1000 if a request stalls
  
  Niko1221
  and
  claude
  committed
  6f32ec0
Commits on Oct 5, 2026
- cpu: IQ3_S / IQ2_S AVX2 decode without the per-index shift/mask/or (1.07-1.6x per row)
  sanastasiou
  committed
  9233f99

Commits

Commits on Oct 2, 2026

Commits on Oct 3, 2026