Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Support for MatMul with broadcasting (eg. 3Dx2D matmul) for Xnnpack execution provider #24107

Open
pl121lp opened this issue Mar 19, 2025 · 0 comments
Labels
ep:Xnnpack issues related to XNNPACK EP feature request request for unsupported feature or enhancement

Comments

@pl121lp
Copy link

pl121lp commented Mar 19, 2025

Describe the feature request

Currently the MatMul op can be mapped to XnnPack fully connected kernels. However, only 2Dx2D matrix multiply is supported.
Now, many models have Matmul layers where the input is a 3D tensor, which requires broadcasting to apply matmul 2D and generate the resulting 3D tensor.
Since this is not supported by the Xnnpack execution provider execution falls back on MLAS implementation which can handle this case. ( sgemm.cpp:MlasGemmPackB() )
From what I gather it would be possible to add broadcasting functionality in the Xnnpack execution provider layer?

Describe scenario use case

Support for models that have non 2Dx2D matmul to comply with ONNX spec for the MatMul operation.
See image of layer cutout from a model

Image

@pl121lp pl121lp added the feature request request for unsupported feature or enhancement label Mar 19, 2025
@github-actions github-actions bot added the ep:Xnnpack issues related to XNNPACK EP label Mar 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ep:Xnnpack issues related to XNNPACK EP feature request request for unsupported feature or enhancement
Projects
None yet
Development

No branches or pull requests

1 participant