Repository navigation

More accurate Q4_0 and Q4_1 quantizations - #896

Closed

ikawrakow wants to merge 12 commits into

quantize_experiments

Closed

More accurate Q4_0 and Q4_1 quantizations#896
ikawrakow wants to merge 12 commits into
masterfrom
quantize_experiments

Commits on Apr 11, 2023
- Use better conversion to ints
  Iwan Kawrakow
  committed
  126b984
- Adding new functions for Q4_0 and Q4_1 quantization
  Iwan Kawrakow
  committed
  0c9a967
- Remove forgotten remnant from a discarded change to ggml.c
  Iwan Kawrakow
  committed
  8b3d1f9
- Add ability to use new quantization in quantize-stats
  Iwan Kawrakow
  committed
  92408cd
- Add new quantization to quantize
  Iwan Kawrakow
  committed
  709d235
- Reverting round() change so we can pass tests
  Iwan Kawrakow
  committed
  b6df974
- Improve Q4_0 MSE
  Iwan Kawrakow
  committed
  931ae36
Commits on Apr 12, 2023
- Further improve Q4_0 MSE
  Iwan Kawrakow
  committed
  6bfb00a
- Various experiments, including 5-bit qunatization
  Iwan Kawrakow
  committed
  29b83e5
- POC: Even lower rmse 4-bit Q4_0 quantization
  Iwan Kawrakow
  committed
  679e1cb
Commits on Apr 13, 2023
- POC: Q4_1 for groups of 16 weight
  Iwan Kawrakow
  committed
  6f34961
- POC: Measure rmse of 8 bit quantization
  Iwan Kawrakow
  committed
  97d7ac7