Repository navigation

INTEL B70 32gb support - #423

Closed

maxfridbe wants to merge 73 commits into

Niko1221:mainfrom

maxfridbe:b70

Closed

INTEL B70 32gb support#423
maxfridbe wants to merge 73 commits into
Niko1221:mainfrom
maxfridbe:b70

Commits on Sep 29, 2026
- serve: a llama.cpp engine behind the same boundary (--engine llama)
  
  maxfridbe
  and
  claude
  committed
  2e4c95e
- setup: an Intel Arc path (no CUDA), and the run script starts llama-server itself
  
  maxfridbe
  and
  claude
  committed
  43cd6bd
- docs/INTEL.md: --port and --host, learned the hard way (8080 is open-webui here)
  
  maxfridbe
  and
  claude
  committed
  8e1354b
- README: the Intel Arc port and the rig it was tested on, up top
  
  maxfridbe
  and
  claude
  committed
  5665315
- gitignore: start-llama.sh, generated by setup on the Intel path
  
  maxfridbe
  and
  claude
  committed
  bcfcf40
- docs/INTEL.md: how much context fits on a 32 GB card, measured
  
  maxfridbe
  and
  claude
  committed
  34cae9e
- docs: 128k context proven on the B70 - a 105k-token prompt read at 424 tok/s
  
  maxfridbe
  and
  claude
  committed
  0741381
- sycl: the CUDA engine migrated to SYCL - builds with oneAPI, 19 of 22 kernel parity tests pass on the Arc Pro B70
  
  maxfridbe
  and
  claude
  committed
  7021c80
- sycl: first light - the SYCL engine generates correct text on the Arc Pro B70
  
  maxfridbe
  and
  claude
  committed
  35b58f9
- sycl: ~13 tok/s steady state, coherent 64-token output; window timing, eager stage profiler, docs
  
  maxfridbe
  and
  claude
  committed
  8c789e6
- sycl: AOT build (bmg-g31) - 15.2 tok/s decode, 28.8 tok/s prompt, first token in 0.85 s, no JIT
  
  maxfridbe
  and
  claude
  committed
  3600536
- sycl: multi-entry expert dots - the resident-expert kernels 2.3x faster on iq3_xxs layers
  
  maxfridbe
  and
  claude
  committed
  21c3f48
Commits on Sep 30, 2026
- sycl: fused multi-token router for 256 experts (was 512-only); 16.95 tok/s decode
  
  maxfridbe
  and
  claude
  committed
  36aa5bc
- sycl: clean exit, per-kernel profiling, Q6_K unpack hoisted; measured: the mmvq kernels run at 110-130 GB/s warm
  
  maxfridbe
  and
  claude
  committed
  c382f84
- sycl: long prompts - 693 tok/s at 2k tokens; the streaming source is a thread-safe 512-slot keyed cache
  
  maxfridbe
  and
  claude
  committed
  6e02add
- sycl: Q6_K 16-byte-load kernel (wide projections only), mmvq bench correctness check; bandwidth probe
  
  maxfridbe
  and
  claude
  committed
  20e5b1a
- docs: like-for-like against llama.cpp on the same prompt - prompt 560 vs 138-319 tok/s, decode 20.8 vs 24-26
  
  maxfridbe
  and
  claude
  committed
  8303db2
- docs: 45.3 tok/s decode with the base model's MTP draft layer (87% accepted) - 1.8x llama.cpp on the same card
  
  maxfridbe
  and
  claude
  committed
  511ae75
- docs: 42.7 tok/s decode at the 2,185-token context with MTP (prompt 475 tok/s in the same run)
  
  maxfridbe
  and
  claude
  committed
  52fb638
- Merge remote-tracking branch 'origin/main' into b70
  maxfridbe
  committed
  6a7c357
- sycl: refresh the port for upstream 0.1.25-0.1.27 (re-migration merge), AOT divide rounding, hybrid K8V4 KV
  
  maxfridbe
  and
  claude
  committed
  c3bb85b
- docs: INTEL.md - the merged port's numbers on the B70 (46.5 / 38.3 tok/s decode, 566 tok/s prompt)
  
  maxfridbe
  and
  claude
  committed
  b7d88b9
- sycl: the speed round - warm-up captures, commit/draft overlap, vector-store dequant, XMX kernels (opt-in), profiles
  
  maxfridbe
  and
  claude
  committed
  bf804a1
- sycl: iq4_nl dequant as runs of 8 (0.024 -> 0.017 ms per expert), host timers around the prompt path's per-layer grouping
  
  maxfridbe
  and
  claude
  committed
  3249953
- sycl: PLE rows - a 64-deep read queue and a short first prompt chunk (2,184 tokens 711 -> 765 tok/s, 8,000 857 -> 986)
  
  maxfridbe
  and
  claude
  committed
  cbb4728
- docs: INTEL.md - the 80,000-token prompt: the VRAM plan trap, expert streaming per chunk, where the 77 s go
  
  maxfridbe
  and
  claude
  committed
  66699fb
- docs: INTEL.md - 80k tokens at 1,065 tok/s with 12 staging threads; the SSD is the floor of the streamed experts
  
  maxfridbe
  and
  claude
  committed
  51a9249
- Merge remote-tracking branch 'origin/main' into b70
  maxfridbe
  committed
  35a147f
- sycl: refresh for upstream 0.1.28-0.1.29, fix the streamed-expert prompt path (stale ring pointers, stream-all hang)
  
  maxfridbe
  and
  claude
  committed
  e3e97c9
- sycl: lend cache slots to the prompt path above a 32K context - 80,000 tokens at 1,062 tok/s, decode 35-40 after
  
  maxfridbe
  and
  claude
  committed
  fc8a4de
- sycl: XMX prompt attention v2 (1.5x v1, k8v4 parity complete), long-context x KV-type table
  
  maxfridbe
  and
  claude
  committed
  80be39e
- sycl: STRATA_DUMP_SEL (the prompt path's selected cells); INTEL.md - XMX v2 in the full matrix, selection overlap
  
  maxfridbe
  and
  claude
  committed
  7f9a502
- sycl: serve the port - strata-sycl.sh runs the engine in the oneAPI image as server.py's exe (36 tok/s sampled)
  
  maxfridbe
  and
  claude
  committed
  4fa86aa
- setup: Strata's SYCL engine on Intel cards when it is built; sycl: clean exit in serve mode
  
  maxfridbe
  and
  claude
  committed
  b40bf7b
- setup/docs: KV streaming for the SYCL engine from 64K - 256K context decodes at 31 tok/s (from 4-5)
  
  maxfridbe
  and
  claude
  committed
  a574bff
- sycl: pinned host mirror for experts missing from VRAM, read by the device plan
  
  maxfridbe
  and
  claude
  committed
  1f79044
Commits on Oct 1, 2026
- sycl: SIMD16 variants of the decode mmvq kernels (opt-in STRATA_MMVQ_SG), mmvq_sg_bench
  
  maxfridbe
  and
  claude
  committed
  8aca6c8
- sycl: aligned loads for the Q6_K decode kernel (decode 45 -> 54 tok/s); PLE gather trace
  
  maxfridbe
  and
  claude
  committed
  38a979d
- sycl: decode 54 -> 62.6 tok/s: parallel resident plan, untiled GR down, wide Q4_K/Q5_K/IQ4_XS
  
  maxfridbe
  and
  claude
  committed
  882764d
- sycl: wide Q8_0/IQ4_NL decode kernels, aligned Q6_K activation loads (decode 63.4 -> 65.6 tok/s)
  
  maxfridbe
  and
  claude
  committed
  d119113
- sycl: GR down projection sliced by columns (decode 65.5 -> 70.6 tok/s)
  
  maxfridbe
  and
  claude
  committed
  dcf251c
- serve: log each request's client; SYCL wrapper without the image's health check
  
  maxfridbe
  and
  claude
  committed
  0800340
- docs/INTEL.md: current numbers up front (SYCL port 70.8 tok/s decode, 799 tok/s prompt)
  
  maxfridbe
  and
  claude
  committed
  288d57b
- sycl: IQ4_NL expert dots with the codebook in registers and aligned loads (decode 70.8 -> 76.1 tok/s)
  
  maxfridbe
  and
  claude
  committed
  20a2ccf
- docs/INTEL.md: planned item 9, the original IQ2_XS with the expert overflow in the host mirror
  
  maxfridbe
  and
  claude
  committed
  89e4de4
- sycl: GR norm per (token, stream), aligned loads in the IQ3_XXS/IQ3_S/IQ2_S expert dots
  
  maxfridbe
  and
  claude
  committed
  03c034e
- docs/INTEL.md: re-measured on the current build, long contexts included
  
  maxfridbe
  and
  claude
  committed
  381373c
- docs/INTEL.md: k8v4 long-context rows and the API figure on the current build
  
  maxfridbe
  and
  claude
  committed
  6e6ca04
- docs/INTEL.md: the original Qwen3.8-Flash-Next IQ2_XS runs on the B70 (51-61 tok/s)
  
  maxfridbe
  and
  claude
  committed
  d9ebe6e
- serve: Intel Arc readings in the Monitor tab; a model menu for hosts that swap models
  
  maxfridbe
  and
  claude
  committed
  717e7e8
- Merge upstream main (engine 0.1.31) into b70
  
  maxfridbe
  and
  claude
  committed
  f6c30c0
- PR hygiene: licenses for the vendored code, a generic Intel GPU sampler, no generated files
  
  maxfridbe
  and
  claude
  committed
  8bbefac
- sycl: refresh the port to upstream 0.1.31
  
  maxfridbe
  and
  claude
  committed
  4951fd0
- docs/INTEL.md: lanes per row of the expert kernels measured per verify round (no difference)
  
  maxfridbe
  and
  claude
  committed
  3ed36e0
- sycl: parity tests wait like cudaMemcpy; drain the refill ring in long prompts
  
  maxfridbe
  and
  claude
  committed
  46b366b
- Merge upstream main (0.1.32) into b70; Intel code kept out of shared files
  
  maxfridbe
  and
  claude
  committed
  33706a3
- sycl: port upstream 0.1.32 (re-migration 3-way merge)
  
  maxfridbe
  and
  claude
  committed
  35809f8
Commits on Oct 2, 2026
- sycl: fix the prompt-slot borrow hang (stager event wait under L0 v2); borrow above 32K again
  
  maxfridbe
  and
  claude
  committed
  bf8e11d
- Merge upstream main (0.1.33) into b70
  
  maxfridbe
  and
  claude
  committed
  a7ef65f
- sycl: port upstream 0.1.33 (re-migration of its six engine files)
  
  maxfridbe
  and
  claude
  committed
  26f8056
- sycl: read streamed experts from the right shard; fix IQ2_XS products (Swift 1.5 runs)
  
  maxfridbe
  and
  claude
  committed
  3ee292a
- sycl: pipeline the expert cache's profile fill (model load 3-4x faster)
  
  maxfridbe
  and
  claude
  committed
  57bd713
- Merge upstream main (0.1.35) into b70
  
  maxfridbe
  and
  claude
  committed
  e5c2836
- sycl: port upstream 0.1.35 (re-migration of its changed engine files)
  
  maxfridbe
  and
  claude
  committed
  41ad902
- sycl: port open upstream PRs #385 #463 #453 #374 #363 #407 #413 onto 0.1.35
  
  maxfridbe
  and
  claude
  committed
  88c3ace
- sycl: OpenAI logprobs on /v1/chat/completions (engine LP lines + server_intel)
  
  maxfridbe
  and
  claude
  committed
  4ba35fd
Commits on Oct 3, 2026
- Merge upstream main (0.1.38) into b70
  
  maxfridbe
  and
  claude
  committed
  a3277e3
- sycl: port upstream 0.1.38
  
  maxfridbe
  and
  claude
  committed
  daf3b7e
- sycl: the stream-all prompt walk by default when the VRAM holds >90% of the experts
  
  maxfridbe
  and
  claude
  committed
  96ccd2a
- SYCL: layer split reads each card's free memory; the launcher exposes every card for a split
  
  maxfridbe
  and
  claude
  committed
  1e365a3
- docs: performance numbers in INTEL_PERFORMANCE.md; benchy v1, the standard Intel bench
  
  maxfridbe
  and
  claude
  committed
  f530f9f
- docs: benchy v1 on the B70 with the served configs
  
  maxfridbe
  and
  claude
  committed
  25277f9
Commits on Oct 4, 2026
- docs/INTEL.md: accuracy pass; add the probe programs and rank_kernels.py it cites
  
  maxfridbe
  and
  claude
  committed
  9803183

Commits

Commits on Sep 29, 2026

Commits on Sep 30, 2026

Commits on Oct 1, 2026

Commits on Oct 2, 2026

Commits on Oct 3, 2026

Commits on Oct 4, 2026