Quantize DeepSeek-Coder-V2-Instruct to W8A8(INT8)? #42
Description
Because old GPUs does not support fp8. So, any plan to quantize DeepSeek-Coder-V2-Instruct to W8A8(INT8)?
{{ message }}
Because old GPUs does not support fp8. So, any plan to quantize DeepSeek-Coder-V2-Instruct to W8A8(INT8)?