Repository navigation

feat(kvcache): add reliable quantized KV cache - #309

Open

plsgivemeachane wants to merge 7 commits into

FlashML-org:mainfrom

plsgivemeachane:feat/reliable-quantized-kv-cache

Open

feat(kvcache): add reliable quantized KV cache#309
plsgivemeachane wants to merge 7 commits into
FlashML-org:mainfrom
plsgivemeachane:feat/reliable-quantized-kv-cache

Commits on Aug 31, 2026
- feat(kvcache): sub-byte Q4_0 KV cache quantization
  
  fangyuan-3149
  authored and
  plsgivemeachane
  committed
  c086b28
- fix(kvcache): wire --kv-cache-dtype through the engine
  
  fangyuan-3149
  authored and
  plsgivemeachane
  committed
  99bd90c
- fix(tests): correct sub-byte test expectations and dtype mismatches
  
  fangyuan-3149
  authored and
  plsgivemeachane
  committed
  09c9d94
- fix(kvcache): correct quantized KV attention data path
  plsgivemeachane
  committed
  aad9a16
- fix(attention): keep sink normalizer in fp32
  plsgivemeachane
  committed
  716c1a2
Commits on Sep 1, 2026
- fix(cli): allow KV dtype with capacity overrides
  plsgivemeachane
  committed
  cccac24
Commits on Sep 3, 2026
- Merge branch 'FlashML-org:main' into feat/reliable-quantized-kv-cache
  plsgivemeachane
  authored
  899bf87