Repository navigation
Add weight quantization #321
Description
Weight quantization enables low-power edge devices to perform machine learning by trading a couple of percentage points of prediction accuracy for a very dramatic reduction in computation times.
Mobile devices will motivate dramatic optimizations and code changes to enable cross platform SIMD across very different devices. An alternative is to create an implementation that is fast(simple) enough and correct enough that most users never need to care about numerical performance.
The approach was initially made popular by XNOR-NET for real time edge vision classification.
https://arxiv.org/pdf/1603.05279.pdf
But this is a generalized approach and there have been deeper analyses of the best way to negotiate the bitwidth-vs-model-accuracy scale. The training side can compress until the accuracy drops too low as part of the automatic hyperparameter tuning. This means a slight increase in training time in order to achieve low-power, high-speed streaming inference on edge processors.
Activity
alexanderkyte commentedon Jun 6, 2018
https://arxiv.org/abs/1706.02379
https://arxiv.org/abs/1609.07061
https://arxiv.org/abs/1712.05877
Are my favorite papers on the topic
alexanderkyte commentedon Jun 6, 2018
The indispensable user story of this function is the always-on network. After a low precision network activates, a slower one can double check the answer.
alexanderkyte commentedon Jun 7, 2018
https://www.tensorflow.org/performance/quantization (https://petewarden.com/2016/05/03/how-to-quantize-neural-networks-with-tensorflow/comment-page-1/) is a nicer introduction for the layman
Ivanidzo4ka commentedon Oct 18, 2018
DRI RESPONSE: We don't support NN as learner, but we do support TensorFlow scoring. So if I correctly understand issue, we technically have support for that as well.
I'm intent to close this issue within few days if no one objects.
Ova Multi class codegen support (dotnet#321)
code migration from machinelearning-automl to machinelearning repo fe…
Ova Multi class codegen support (dotnet#321)
Added onnx export functionality for LpNormNormalizingTransformer (#4161)
Added Onnx export functionality to PCATransformer (#4188)