Pytorch function vs pytorch module function

[PCPJ]

Issue Type

Bug

Source

pip (model-compression-toolkit)

MCT Version

2.4.4

OS Platform and Distribution

Ubuntu 24.04

Python version

3.12

Describe the issue

I was trying to use the post training quantization with my model, and I faced this problem.
When using argmax to get the max conf score and the class id (detection model), the end
of my model forward function was looking like this:

scores, class_ids = torch.argmax(conf_scores, dim=-1, keepdim=True)
outputs = [
    boxes,
    scores,
    class_ids.float(),
]
final = torch.cat(outputs, dim=-1)
return final

Tensor class_ids is long type, I need to make it float to concatenate with other tensors.
This float conversion was getting me this error message:

ERROR:__main__:❌ Error: 'torch.dtype' object has no attribute '__name__'
ERROR:__main__:Complete Stack trace:
Traceback (most recent call last):
  File "/home/pcpj/1TBHD/luna/ML-CV2/demo/export_imx_model.py", line 91, in main
    mct.ptq.pytorch_post_training_quantization(
  File "/home/pcpj/miniforge3/envs/ML-CV2/lib/python3.12/site-packages/model_compression_toolkit/ptq/pytorch/quantization_facade.py", line 125, in pytorch_post_training_quantization
    tg, bit_widths_config, _, scheduling_info = core_runner(in_model=in_module,
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pcpj/miniforge3/envs/ML-CV2/lib/python3.12/site-packages/model_compression_toolkit/core/runner.py", line 99, in core_runner
    graph = graph_preparation_runner(in_model,
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pcpj/miniforge3/envs/ML-CV2/lib/python3.12/site-packages/model_compression_toolkit/core/graph_prep_runner.py", line 73, in graph_preparation_runner
    graph = read_model_to_graph(in_model,
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pcpj/miniforge3/envs/ML-CV2/lib/python3.12/site-packages/model_compression_toolkit/core/graph_prep_runner.py", line 214, in read_model_to_graph
    graph.set_fqc(fqc)
  File "/home/pcpj/miniforge3/envs/ML-CV2/lib/python3.12/site-packages/model_compression_toolkit/core/common/graph/base_graph.py", line 149, in set_fqc
    is_node_in_fqc = any([n.is_match_type(_type) for _type in fqc_layers]) or \
                          ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pcpj/miniforge3/envs/ML-CV2/lib/python3.12/site-packages/model_compression_toolkit/core/common/graph/functional_node.py", line 89, in is_match_type
    names_match = _type.__name__ == self.type.__name__
                                    ^^^^^^^^^^^^^^^^^^
AttributeError: 'torch.dtype' object has no attribute '__name__'. Did you mean: '__ne__'?

I dug a bit inside the graph generation and I believed I found the problem.
Here is the solution I used. It is on this segment.

if node.op == CALL_METHOD:
    # Node is a PyTorch function such as torch.add, torch.reshape etc.
    if hasattr(torch, node.target):
        node_type = getattr(torch, node.target)
    elif hasattr(torch.Tensor, node.target):
        node_type = getattr(torch.Tensor, node.target)
        if node_type==torch.Tensor.to:
            Logger.critical(f"The call method \"to\" is not supported. Please consider moving \"torch.Tensor.to\" operations to init code.")  # pragma: no cover
    else:
        Logger.critical(f"The call method '{node.target}' in {node} is not supported.")  # pragma: no cover
    if hasattr(torch, node.target) and hasattr(torch.Tensor, node.target):
        Logger.warning(f"Method '{node.target}' exists both as a torch function and a torch.Tensor method. "  # pragma: no cover
                        f"Currently, MCT will use the one that is callable."
                        f"If both are callable, MCT will use the torch function.")  # pragma: no cover
        iscallable_torch = callable(getattr(torch, node.target))
        iscallable_torch_tensor = callable(getattr(torch.Tensor, node.target))
        if iscallable_torch:
            node_type = getattr(torch, node.target)
        elif iscallable_torch_tensor:
            node_type = getattr(torch.Tensor, node.target)
        else:
            Logger.critical(f"Method '{node.target}' is callable neither in torch nor in torch.Tensor. You WILL get a runtime error later.")  # pragma: no cover

Summarizing, the .float() method exists inside of torch and torch.Tensor, but the current implementation of MCT is giving preference to torch, which is not my case. My solution does not handles well the case of both torch and torch.Tensor having the function name, and both being callable. Don`t even know if such case is possible.
In the end it did not helped me much, because my end objective is to run my model on Sony IMX500 and apparently it does not support int-float conversion.

Hope it helps. Thanks.

Expected behaviour

A quantized model.

Code to reproduce the issue

bit_cfg = mct.core.BitWidthConfig()
tpc = get_target_platform_capabilities(tpc_version="4.0", device_type="imx500")
mct_config = mct.core.CoreConfig(
    mixed_precision_config=mct.core.MixedPrecisionQuantizationConfig(
        num_of_images=10
    ),
    quantization_config=mct.core.QuantizationConfig(
        concat_threshold_update=True
    ),
    bit_width_config=bit_cfg,
)

num_calibration_batches = 10

def repr_datagen():
    for _ in range(num_calibration_batches):
        array = np.random.random((8, 3, 320,320))
        yield [array]

logger.info("Starting quantization...")
quantized_module, quantization_info = (
    mct.ptq.pytorch_post_training_quantization(
        in_module=model,
        representative_data_gen=repr_datagen,
        core_config=mct_config,
        target_platform_capabilities=tpc,
    )
)

Log output

No response

[YingYang-YY]

@PCPJ Thank you for raising the issue. We are now investigating the issue.