Skip to content

torch.export fails when model uses ImageList.from_tensors #5347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dgcnz opened this issue Aug 13, 2024 · 2 comments · May be fixed by #5467
Closed

torch.export fails when model uses ImageList.from_tensors #5347

dgcnz opened this issue Aug 13, 2024 · 2 comments · May be fixed by #5467

Comments

@dgcnz
Copy link

dgcnz commented Aug 13, 2024

Instructions To Reproduce the 🐛 Bug:

  1. Full runnable code or full changes you made:
from detectron2.structures import ImageList
from torch.export import export
import torch

class M(torch.nn.Module):
    def forward(self, images: list[torch.Tensor]):
        images = ImageList.from_tensors(images)
        return images.tensor
    

example_kwargs = {
    "images": list(torch.randn(1, 3, 24, 24)),
}
exported_program: torch.export.ExportedProgram = export(
    M(), (), kwargs=example_kwargs
)
  1. What exact command you run:
  2. Full logs or other relevant observations:
GuardOnDataDependentSymNode: Could not guard on data-dependent expression u1 < 0 (unhinted: u1 < 0).  (Size-like symbols: none)

Potential framework code culprit (scroll up for full backtrace):
  File ".venv/lib/python3.10/site-packages/torch/_refs/__init__.py", line 2849, in constant_pad_nd
    if pad[pad_idx + 1] < 0:
  1. please simplify the steps as much as possible so they do not require additional resources to
    run, such as a private dataset.

Expected behavior:

If there are no obvious error in "full logs" provided above,
please tell us the expected behavior.

Environment:

-------------------------------  ----------------------------------------------------------------------------------------------
sys.platform                     darwin
Python                           3.10.12 (main, Dec  4 2023, 21:38:54) [Clang 14.0.3 (clang-1403.0.22.14.1)]
numpy                            1.26.4
detectron2                       0.6 @/Users/dgcnz/development/amsterdam/edge/detrex/detectron2/detectron2
Compiler                         clang 15.0.0
CUDA compiler                    not available
DETECTRON2_ENV_MODULE            <not set>
PyTorch                          2.3.1 @/Users/dgcnz/development/amsterdam/edge/.venv/lib/python3.10/site-packages/torch
PyTorch debug build              False
torch._C._GLIBCXX_USE_CXX11_ABI  False
GPU available                    No: torch.cuda.is_available() == False
Pillow                           10.4.0
torchvision                      0.18.1 @/Users/dgcnz/development/amsterdam/edge/.venv/lib/python3.10/site-packages/torchvision
fvcore                           0.1.5.post20221221
iopath                           0.1.9
cv2                              4.10.0
-------------------------------  ----------------------------------------------------------------------------------------------
PyTorch built with:
  - GCC 4.2
  - C++ Version: 201703
  - clang 14.0.3
  - OpenMP 201811
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: NO AVX
  - Build settings: BLAS_INFO=accelerate, BUILD_TYPE=Release, CXX_COMPILER=/Applications/Xcode_14.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++, CXX_FLAGS= -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_PYTORCH_METAL_EXPORT -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DUSE_COREML_DELEGATE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=braced-scalar-init -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wvla-extension -Wsuggest-override -Wnewline-eof -Winconsistent-missing-override -Winconsistent-missing-destructor-override -Wno-pass-failed -Wno-error=pedantic -Wno-error=old-style-cast -Wno-error=inconsistent-missing-override -Wno-error=inconsistent-missing-destructor-override -Wconstant-conversion -Wno-invalid-partial-specialization -Wno-missing-braces -Qunused-arguments -fcolor-diagnostics -faligned-new -Wno-unused-but-set-variable -fno-math-errno -fno-trapping-math -Werror=format -DUSE_MPS -Wno-unused-private-field -Wno-missing-braces, LAPACK_INFO=accelerate, TORCH_VERSION=2.3.1, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EIGEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=OFF, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF,
@lodgegao
Copy link

Met this problem as well, wondering how to fix it

@yushangdi
Copy link
Contributor

yushangdi commented May 15, 2025 •
edited
Loading

I tried on the nightly branch of pytroch and can reproduce the GuardOnDataDependentSymNode error (I got a different GuardOnDataDependentSymNode error than the one in the orignal post though. ) To fix:

Option 1 is you can use draft_export, like

exported_program: torch.export.ExportedProgram = draft_export(
    M(), (), kwargs=example_kwargs
)

Option 2, which is the proper fix, is you need to add some torch._check calls in detectron2 code to give export some hint on dynamic shapes.

The fix you need is #5467

        if len(tensors) == 1:
            # This seems slightly (2%) faster.
            # TODO: check whether it's faster for multiple images as well
            image_size = image_sizes[0]
            u0 = ( max_size[-1] - image_size[1]).item()
            u1 = ( max_size[-2] - image_size[0]).item()

            padding_size = [0, u0, 0, u1]
            torch._check(u0 >= 0)
            torch._check(u1 >= 0)
            batched_imgs = F.pad(tensors[0], padding_size, value=pad_value).unsqueeze_(0)

cc @angelayi @pianpwk

@yushangdi yushangdi mentioned this issue May 15, 2025
Open
yushangdi added a commit to yushangdi/detectron2 that referenced this issue May 16, 2025
@yushangdi @facebook-github-bot
Add torch._check hints for torch.export
a3bae30 
Summary:
Fixes facebookresearch#5347

This allows for torch.export.export ImageList.from_tensors.

Differential Revision: D74835454
@yushangdi yushangdi mentioned this issue May 16, 2025
Closed
yushangdi added a commit to yushangdi/detectron2 that referenced this issue May 16, 2025
@yushangdi @facebook-github-bot
Add torch._check hints for torch.export
ec52805 
Summary:

Fixes facebookresearch#5347

This allows for torch.export.export ImageList.from_tensors.

Differential Revision: D74835454
yushangdi added a commit to yushangdi/detectron2 that referenced this issue May 16, 2025
@yushangdi @facebook-github-bot
Add torch._check hints for torch.export
1ced578 
Summary:

Fixes facebookresearch#5347

This allows for torch.export.export ImageList.from_tensors.

Differential Revision: D74835454
yushangdi added a commit to yushangdi/detectron2 that referenced this issue May 16, 2025
@yushangdi @facebook-github-bot
Add torch._check hints for torch.export
2f8229d 
Summary:

Fixes facebookresearch#5347

This allows for torch.export.export ImageList.from_tensors.

Differential Revision: D74835454
yushangdi added a commit to yushangdi/detectron2 that referenced this issue May 20, 2025
@yushangdi @facebook-github-bot
Add torch._check hints for torch.export
39f030d 
Summary:

Fixes facebookresearch#5347

This allows for torch.export.export ImageList.from_tensors.

Differential Revision: D74835454
yushangdi added a commit to yushangdi/detectron2 that referenced this issue May 21, 2025
@yushangdi @facebook-github-bot
Add torch._check hints for torch.export
bba1286 
Summary:

Fixes facebookresearch#5347

This allows for torch.export.export ImageList.from_tensors.

Differential Revision: D74835454
yushangdi added a commit to yushangdi/detectron2 that referenced this issue May 21, 2025
@yushangdi @facebook-github-bot
Add torch._check hints for torch.export
af6f31c 
Summary:

Fixes facebookresearch#5347

This allows for torch.export.export ImageList.from_tensors.

Differential Revision: D74835454
yushangdi added a commit to yushangdi/detectron2 that referenced this issue May 21, 2025
@yushangdi @facebook-github-bot
Add torch._check hints for torch.export
a1f1c7d 
Summary:
Pull Request resolved: facebookresearch#5468

Fixes facebookresearch#5347

This allows for torch.export.export ImageList.from_tensors.

Differential Revision: D74835454
facebook-github-bot pushed a commit that referenced this issue May 22, 2025
@yushangdi @facebook-github-bot
Add torch._check hints for torch.export
65184fc 
Summary:
Pull Request resolved: #5468

Fixes #5347

This allows for torch.export.export ImageList.from_tensors.

Reviewed By: wat3rBro

Differential Revision: D74835454

fbshipit-source-id: 0da4c30cb7fdebcab60313e43a41f73f21533d72
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@yushangdi @dgcnz @lodgegao