All pull requests

Pull requests

• fix(fp8): per-row dynamic FP8 activation scales

  #99·gdevenyi opened Oct 21, 2026Owner·

  ·

• perf(qsa): score prefill rows in tiles of 16 query rows per program

  #98·gdevenyi opened Oct 21, 2026Owner·

  ·

• perf(engine): FREETOKEN_GRAPH_UPLOAD switch; #697's capture-time upload costs 1-2% decode

  #97·gdevenyi opened Oct 21, 2026Owner·

  ·

• feat(scheduler): --logprob-read-isolate makes logprob reads independent of other traffic

  #96·gdevenyi opened Oct 20, 2026Owner·

  ·

• 5080deploy round 7: rebuild on cc56b6bd, AnyJev scoring fields, #718/#749/#710 fixes, fp32 NVFP4 globals

  #95·gdevenyi opened Oct 20, 2026Owner·

  ·

• 5080deploy: OpenAI compat: stream/non-stream parser parity, chat logprobs under the parsers, logprobs/usage fixes, cache report + decode interleave

  #92·gdevenyi opened Oct 15, 2026Owner·

  ·
  1

• 5080deploy: rebuild on upstream main 0bca88f7 (#689 slot pools; identical decode, VRAM and greedy output)

  #91·gdevenyi opened Oct 15, 2026Owner·

  ·

• 5080deploy: upstream #676 on-demand short prefills (+ two fixes), 1080deploy and chatdnp picks

  #86·gdevenyi opened Oct 13, 2026Owner·

  ·
  2

• 5080deploy: rebuild on upstream main 9c635a15 (torch 2.14 + NVFP4 QSA spill fix, -20% cold TTFT)

  #85·gdevenyi opened Oct 13, 2026Owner·

  ·

• 5080deploy: rebuild on upstream main 07813248 (upstream #631 marlin decode, +3% decode; nothing superseded)

  #84·gdevenyi opened Oct 8, 2026Owner·

  ·

• 5080deploy: merge upstream #614 and #593, read compressed-tensors NVFP4 experts for qwen4_exp

  #82·gdevenyi opened Oct 7, 2026Owner·

  ·
  1

• 5080deploy: rebuild on upstream main 377a3bc (drops the upstream-merged #559 and a dead merge/revert pair)

  #81·gdevenyi opened Oct 7, 2026Owner·

  ·

• 5080deploy: merge upstream #578, NVFP4 prefill dequant by bit placement (-32% TTFT at 64K, bit-identical)

  #79·gdevenyi opened Oct 2, 2026Owner·

  ·

• 5080deploy: generic fixes from 1080deploy (PLE per-fill readback events, host-tier probes, GDN workspace reserve, server fixes, --embed-weights host off)

  #77·gdevenyi opened Oct 2, 2026Owner·

  ·
  4

• 5080deploy: merge upstream #559, #531, #269, #468, #461, #484, #464, #266, #572

  #76·gdevenyi opened Oct 2, 2026Owner·

  ·
  8

• 5080deploy: rebuild on upstream main d3512b4 (upstream #546 max_tokens fix, #548 row store, #588 eager PLE buffers without its graph fence)

  #75·gdevenyi opened Oct 2, 2026Owner·

  ·
  3

• 5080deploy: expert prefetch into low-priority slots with single-writer slot maps (FREETOKEN_MOE_PREFETCH), +22% decode

  #74·gdevenyi opened Sep 27, 2026Owner·

  ·
  1

• 5080deploy: opt-in slim expert copy kernel for copies beside compute (FREETOKEN_MOE_SLIM_COPY, off)

  #73·gdevenyi opened Sep 26, 2026Owner·

  ·

• 5080deploy: shared expert on a side stream beside the miss copy (FREETOKEN_MOE_COPY_OVERLAP), +2.1% over rule eviction

  #72·gdevenyi opened Sep 26, 2026Owner·

  ·

• 5080deploy: scored expert eviction (--moe-cache-policy kd/kdfb/rule), +5.5% decode

  #71·gdevenyi opened Sep 26, 2026Owner·

  ·

• 5080deploy: opt-in on-demand experts for short prefills (FREETOKEN_MOE_SMALL_PREFILL_TOKENS, off)

  #68·gdevenyi opened Sep 24, 2026Owner·

  ·

• 5080deploy: run the PLE graph fill on its own thread, submitted before the launch (fixes #65)

  #67·gdevenyi opened Sep 24, 2026Owner·

  ·

• 5080deploy: fix 11 bugs found re-reviewing the stack

  #64·gdevenyi opened Sep 24, 2026Owner·

  ·

• 5080deploy: merge upstream #224, OpenAI-compatible logprobs

  #63·gdevenyi opened Sep 24, 2026Owner·

  ·

• 5080deploy: merge upstream #504, per-request metrics under --enable-metrics

  #62·gdevenyi opened Sep 24, 2026Owner·

  ·