[microNPU][ETHOSU] Add fixed point for tanh - #16266
Conversation
ekalda
left a comment
ekalda
left a comment
There was a problem hiding this comment.
Thanks @Aleksei-grovety! I think in the current form of the patch the calculated int16 values are not actually used...
| op_map = { | ||
| "CLIP": vapi.NpuActivationOp.NONE_OR_RELU, | ||
| "TANH": vapi.NpuActivationOp.TABLE_LOOKUP, | ||
| "TANH": vapi.NpuActivationOp.TANH, |
There was a problem hiding this comment.
This change would make it always use the NPU's builtin tanh function instead of the calculated lookup table values, making it to not match TFLite reference kernels and turning all tanh LUT value calculation into dead code.
There was a problem hiding this comment.
These changes have been cancelled and the calculated lookup table values are being used, but first, PR merge is expected
There was a problem hiding this comment.
PR was merged and the code has been updated.
| return identity | ||
| return identity | ||
| elif params.ifm.dtype == "int16": | ||
| lut_tanh = relay.const([], "int16") |
There was a problem hiding this comment.
Seems like this is adding an empty lookup table to the identity operator?
There was a problem hiding this comment.
After the update, the calculated lookup table values for Int16 are used.
09604c9 to
5748beb
Compare
Add support for calculation tanh with 16 bits fixed point. Add flag enable_fixed_point to enable fixed point calculation. We get good accuracy with 1 bit to integer part and 15 bits for fractional, with other cases we get worse results.
use fixed_point_multiply to define fraction_size, use LUT for tanh
5748beb to
de23e58
Compare
| output_max - output_min | ||
| ) | ||
| table_min = np.iinfo(np.int16).min | ||
| table_max = np.iinfo(np.int16).max |
There was a problem hiding this comment.
nit: these results can be used to calculate input_min, input_max etc above
|
Thanks @Aleksei-grovety @ekalda! |
Add support for calculation tanh with 16 bits fixed point (legalization of non-quantized tanh operation with quantization by fixed point multiplication).
3 participants
Add support for calculation tanh with 16 bits fixed point (legalization of non-quantized tanh operation with quantization by fixed point multiplication).
cc @lhutton1, @ekalda, @leandron