Description
Describe the feature request
Describe the bug:
I am able to do quantization with:
quantized_model = quantize_dynamic(
model_input=model_input,
model_output=model_output,
# optimize_model=False,
weight_type=QuantType.QInt8,
# activation_type=QuantType.QInt8,
# op_types_to_quantize=["Conv", "MatMul", "Add","Relu"],
extra_options={"UseSymmetric": True, "ActivationSymmetric": True,
"EnableSubgraph": True, "ForceQuantizeNoInputCheck": True}
)
but I got the following error while doing the inference:
File "/home/pc10/anaconda3/envs/evo-venv/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 465, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/home/pc10/anaconda3/envs/evo-venv/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 537, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.NotImplemented: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Could not find an implementation for ConvInteger(10) node with name 'Conv__116_quant'
System information
OS Platform and Distribution: Ubuntu Ubuntu 20.04.5 LTS(5.15.0-48-generic)
CPU 12th Gen Intel(R) Core(TM) i9-12900K
ONNX Runtime installed by pip:
ONNX Runtime version: 1.20.1
Python version: 3.11.11
How should I use symmetric quantization to quantify weight and obtain the correct quantization model?
Describe scenario use case
quantized_model = quantize_dynamic(
model_input=model_input,
model_output=model_output,
# optimize_model=False,
weight_type=QuantType.QInt8,
# activation_type=QuantType.QInt8,
# op_types_to_quantize=["Conv", "MatMul", "Add","Relu"],
extra_options={"UseSymmetric": True, "ActivationSymmetric": True,
"EnableSubgraph": True, "ForceQuantizeNoInputCheck": True}
)