Setting activation bit width #1649
Replies: 1 comment
-
|
It can be done by setting 'dynamic' flag in quantization config and changing the relevant code to reflect the desired range or no. of bits. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I am trying to simulate few results in the GPTQ v2 paper for which activation is set to 4 bits. I see that 'QuantizeConfig' has a parameter named 'bits' which I suppose for weights. How do we set activation bit width? Thanks.
Beta Was this translation helpful? Give feedback.
All reactions