Could not find any implementation for node /model.22/proto/cv3/conv/Conv + PWN(PWN(/model.22/proto/cv3/act/Sigmoid), /model.22/proto/cv3/act/Mul). failure of TensorRT 8.6.1 when running C++ TensorRT.trtexec on GPU GeForce RTX 4060

[feige211314]

Description

In windows system , I use tensorRT C++ tools trtexec.exe to quantificat yolov8n-seg.onnx fails with error ：

Could not find any implementation for node /model.22/proto/cv3/conv/Conv + PWN(PWN(/model.22/proto/cv3/act/Sigmoid), /model.22/proto/cv3/act/Mul).

My input:
trtexec --onnx=yolov8n-seg.onnx --int8 --saveEngine=yolov8s-seg-int8.engine --calib=none --workspace=4096 --verbose --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640

Use tensorRT 8.6.1 C++ API to quantization is the same error!!!

Use tensorRT 8.6.1 python API to quantization is fine!!!!

I have tried the higher version of tensorRT, but the same error still occurred.

Do the two versions have any different processing methods for the YOLOv8 segmentation model?
I want to use the C++ interface to conduct quantitative analysis.

Could you please help me solve this problem? Thank you very much.

The attachment contains my complete log and code.
yolov8_tensorRT.zip

Environment

TensorRT Version: 8.6.1

NVIDIA GPU: GeForce RTX 4060

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Feb_27_16:28:36_Pacific_Standard_Time_2024
Cuda compilation tools, release 12.4, V12.4.99
Build cuda_12.4.r12.4/compiler.33961263_0

[08/14/2025-18:48:50] [I] [TRT] Input filename: yolov8n-seg.onnx
[08/14/2025-18:48:50] [I] [TRT] ONNX IR version: 0.0.7
[08/14/2025-18:48:50] [I] [TRT] Opset version: 13
[08/14/2025-18:48:50] [I] [TRT] Producer name: pytorch
[08/14/2025-18:48:50] [I] [TRT] Producer version: 2.4.0
[08/14/2025-18:48:50] [I] [TRT] Domain:
[08/14/2025-18:48:50] [I] [TRT] Model version: 0
[08/14/2025-18:48:50] [I] [TRT] Doc string:

MyLog:

(base) C:\Users\Administrator\Desktop\yolov8_tensorRT>trtexec --onnx=yolov8n-seg.onnx --int8 --saveEngine=yolov8s-seg-int8.engine --calib=none --workspace=4096 --verbose --minShapes=images:1x3x640x640 --optShapes=images:4x3x640x640 --maxShapes=images:8x3x640x640 >log2.txt
[08/14/2025-19:27:58] [W] --workspace flag has been deprecated by --memPoolSize flag.
[08/14/2025-19:28:03] [W] [TRT] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor /model.22/Squeeze_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor /model.22/Squeeze_1_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor /model.22/Squeeze_2_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 367) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 368) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 378) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 382) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 383) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 393) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor /model.22/Reshape_6_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor /model.22/Reshape_7_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 457) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 458) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 468) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 472) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 473) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 483) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor /model.22/Reshape_11_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor /model.22/Reshape_12_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 547) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 548) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 558) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 562) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 563) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 573) [Constant]output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor /model.22/Reshape_16_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor /model.22/Reshape_17_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor /model.22/dfl/Softmax_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[08/14/2025-19:28:56] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 739) [Constant]output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
**[08/14/2025-19:33:23] [E] Error[10]: Could not find any implementation for node /model.22/proto/cv3/conv/Conv + PWN(PWN(/model.22/proto/cv3/act/Sigmoid), /model.22/proto/cv3/act/Mul).
[08/14/2025-19:33:23] [E] Error[10]: [optimizer.cpp::nvinfer1::builder::cgraph::LeafCNode::computeCosts::3869] Error Code 10: Internal Error (Could not find any implementation for node /model.22/proto/cv3/conv/Conv + PWN(PWN(/model.22/proto/cv3/act/Sigmoid), /model.22/proto/cv3/act/Mul).)**
[08/14/2025-19:33:23] [E] Engine could not be created from network
[08/14/2025-19:33:23] [E] Building engine failed
[08/14/2025-19:33:23] [E] Failed to create engine from model or file.
[08/14/2025-19:33:23] [E] Engine set up failed

[kevinch-nv]

What was the python API script that you've used? There should be no difference in engine build between the C++ and python APIs.

It's possible that TRT is lacking the INT8 kernels to run this model, can you try with --fp16 and see if the performance increase is sufficient?

[feige211314]

What was the python API script that you've used? There should be no difference in engine build between the C++ and python APIs.

It's possible that TRT is lacking the INT8 kernels to run this model, can you try with --fp16 and see if the performance increase is sufficient?

Thank you very much for your suggestions. The python API that I used was provided by the official YOLOv8 team.
I examined the source code and found that the interface being called was also that of tensorRT.
Additionally, I also tested the use of the C++ API of tensorRT to export the FP16 engine, and it was successful.
I also suspect that tensorRT does not support INT8 operators. However, the Python API of the same version of tensorRT can successfully export, which makes me very puzzled.

This is my script：

from ultralytics import YOLO
import argparse

if name == 'main':
parser = argparse.ArgumentParser(description='Export model to TensorRT')
parser.add_argument('--model', type=str, default='model/yolov8n-seg.pt', required=False, help='Path to the .pt')
parser.add_argument('--q', type=str, default='int8', required=False, help='[fp16, int8]')
parser.add_argument('--data', type=str, default='datasets/coco.yaml', required=False, help='Dataset')
parser.add_argument('--batch', type=int, default=4, required=False, help='batch size')
parser.add_argument('--workspace', type=int, default=4, required=False, help='workspace')
args = parser.parse_args()

# Load a model
model = YOLO(args.model)

# Export the model
if args.q == 'fp16':
    model.export(format = 'engine', data = args.data, batch = args.batch, workspace = args.workspace, half = True, dynamic=True, opset=13)
elif args.q == 'int8':
    model.export(format = 'engine', data = args.data, batch = args.batch, workspace = args.workspace, simplify=True, int8 = True, dynamic=True, opset=13)
elif args.q == 'fp32':
    model.export(format = 'engine', batch = args.batch, dynamic=True)
else:
    print('\n[ERROR] Make sure "--q" parameter is entered correctly!')

[kevinch-nv]

I'm not familiar with the ultralytics code, but it looks here that a calibration cache is saved when int8 is used: https://github.com/ultralytics/ultralytics/blob/23d79250e7420945792362f07b8d818320fdce49/ultralytics/utils/export.py#L219-L221

Can you try loading the saved cache with the c++ API to see if that helps with engine build?