Skip to content

Commit f2e1e6c

Browse files
lanluo-nvidiaperi044
lanluo-nvidia
and
peri044
authored
cherry pick doc fix #3238 from main to release2.5 (#3240)
Co-authored-by: Dheeraj Peri <[email protected]>
1 parent edf63bb commit f2e1e6c

9 files changed

+75
-47
lines changed

‎docsrc/index.rst

Lines changed: 37 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -48,10 +48,35 @@ User Guide
4848
user_guide/saving_models
4949
user_guide/runtime
5050
user_guide/using_dla
51+
52+
53+
Tutorials
54+
------------
55+
56+
* :ref:`torch_compile_advanced_usage`
57+
* :ref:`vgg16_ptq`
58+
* :ref:`engine_caching_example`
59+
* :ref:`engine_caching_bert_example`
60+
* :ref:`refit_engine_example`
61+
* :ref:`serving_torch_tensorrt_with_triton`
62+
* :ref:`torch_export_cudagraphs`
63+
* :ref:`custom_kernel_plugins`
64+
* :ref:`mutable_torchtrt_module_example`
65+
66+
.. toctree::
67+
:caption: Tutorials
68+
:maxdepth: 1
69+
:hidden:
70+
5171
tutorials/_rendered_examples/dynamo/torch_compile_advanced_usage
5272
tutorials/_rendered_examples/dynamo/vgg16_ptq
5373
tutorials/_rendered_examples/dynamo/engine_caching_example
74+
tutorials/_rendered_examples/dynamo/engine_caching_bert_example
5475
tutorials/_rendered_examples/dynamo/refit_engine_example
76+
tutorials/serving_torch_tensorrt_with_triton
77+
tutorials/_rendered_examples/dynamo/torch_export_cudagraphs
78+
tutorials/_rendered_examples/dynamo/custom_kernel_plugins
79+
tutorials/_rendered_examples/dynamo/mutable_torchtrt_module_example
5580

5681
Dynamo Frontend
5782
----------------
@@ -97,27 +122,28 @@ FX Frontend
97122

98123
fx/getting_started_with_fx_path
99124

100-
Tutorials
125+
Model Zoo
101126
------------
102-
* :ref:`torch_tensorrt_tutorials`
103-
* :ref:`serving_torch_tensorrt_with_triton`
127+
* :ref:`torch_compile_resnet`
128+
* :ref:`torch_compile_transformer`
129+
* :ref:`torch_compile_stable_diffusion`
130+
* :ref:`torch_export_gpt2`
131+
* :ref:`torch_export_llama2`
104132
* :ref:`notebooks`
105133

106134
.. toctree::
107-
:caption: Tutorials
135+
:caption: Model Zoo
108136
:maxdepth: 3
109137
:hidden:
110-
111-
tutorials/serving_torch_tensorrt_with_triton
112-
tutorials/notebooks
138+
113139
tutorials/_rendered_examples/dynamo/torch_compile_resnet_example
114140
tutorials/_rendered_examples/dynamo/torch_compile_transformers_example
115141
tutorials/_rendered_examples/dynamo/torch_compile_stable_diffusion
116-
tutorials/_rendered_examples/dynamo/torch_export_cudagraphs
117-
tutorials/_rendered_examples/dynamo/custom_kernel_plugins
118142
tutorials/_rendered_examples/distributed_inference/data_parallel_gpt2
119143
tutorials/_rendered_examples/distributed_inference/data_parallel_stable_diffusion
120-
tutorials/_rendered_examples/dynamo/mutable_torchtrt_module_example
144+
tutorials/_rendered_examples/dynamo/torch_export_gpt2
145+
tutorials/_rendered_examples/dynamo/torch_export_llama2
146+
tutorials/notebooks
121147

122148
Python API Documentation
123149
------------------------
@@ -214,4 +240,4 @@ Legacy Further Information (TorchScript)
214240
* `GTC 2021 Fall Talk <https://www.nvidia.com/en-us/on-demand/session/gtcfall21-a31107/>`_
215241
* `PyTorch Ecosystem Day 2021 <https://assets.pytorch.org/pted2021/posters/I6.png>`_
216242
* `PyTorch Developer Conference 2021 <https://s3.amazonaws.com/assets.pytorch.org/ptdd2021/posters/D2.png>`_
217-
* `PyTorch Developer Conference 2022 <https://pytorch.s3.amazonaws.com/posters/ptc2022/C04.pdf>`_
243+
* `PyTorch Developer Conference 2022 <https://pytorch.s3.amazonaws.com/posters/ptc2022/C04.pdf>`_

‎docsrc/tutorials/notebooks.rst

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,9 @@
11
.. _notebooks:
22

3-
Example notebooks
3+
Legacy notebooks
44
===================
55

6-
There exists a number of notebooks which cover specific using specific features and models
7-
with Torch-TensorRT
6+
There exists a number of notebooks which demonstrate different model conversions / features / frontends available within Torch-TensorRT
87

98
Notebooks
109
------------

‎examples/README.rst

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,4 @@
11
.. _torch_tensorrt_tutorials:
22

33
Torch-TensorRT Tutorials
4-
===========================
5-
6-
The user guide covers the basic concepts and usage of Torch-TensorRT.
7-
We also provide a number of tutorials to explore specific usecases and advanced concepts
4+
===========================

‎examples/dynamo/README.rst

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,22 @@
1-
.. _torch_compile:
1+
.. _torch_tensorrt_examples:
22

3-
Dynamo / ``torch.compile``
4-
----------------------------
3+
Here we provide examples of Torch-TensorRT compilation of popular computer vision and language models.
54

6-
Torch-TensorRT provides a backend for the new ``torch.compile`` API released in PyTorch 2.0. In the following examples we describe
7-
a number of ways you can leverage this backend to accelerate inference.
5+
Dependencies
6+
------------------------------------
87

8+
Please install the following external dependencies (assuming you already have correct `torch`, `torch_tensorrt` and `tensorrt` libraries installed (`dependencies <https://github.com/pytorch/TensorRT?tab=readme-ov-file#dependencies>`_))
9+
10+
.. code-block:: python
11+
12+
pip install -r requirements.txt
13+
14+
15+
Model Zoo
16+
------------------------------------
917
* :ref:`torch_compile_resnet`: Compiling a ResNet model using the Torch Compile Frontend for ``torch_tensorrt.compile``
1018
* :ref:`torch_compile_transformer`: Compiling a Transformer model using ``torch.compile``
11-
* :ref:`torch_compile_advanced_usage`: Advanced usage including making a custom backend to use directly with the ``torch.compile`` API
1219
* :ref:`torch_compile_stable_diffusion`: Compiling a Stable Diffusion model using ``torch.compile``
13-
* :ref:`torch_export_cudagraphs`: Using the Cudagraphs integration with `ir="dynamo"`
14-
* :ref:`custom_kernel_plugins`: Creating a plugin to use a custom kernel inside TensorRT engines
15-
* :ref:`refit_engine_example`: Refitting a compiled TensorRT Graph Module with updated weights
16-
* :ref:`mutable_torchtrt_module_example`: Compile, use, and modify TensorRT Graph Module with MutableTorchTensorRTModule
17-
* :ref:`vgg16_fp8_ptq`: Compiling a VGG16 model with FP8 and PTQ using ``torch.compile``
18-
* :ref:`engine_caching_example`: Utilizing engine caching to speed up compilation times
19-
* :ref:`engine_caching_bert_example`: Demonstrating engine caching on BERT
20+
* :ref:`_torch_export_gpt2`: Compiling a GPT2 model using AOT workflow (`ir=dynamo`)
21+
* :ref:`_torch_export_llama2`: Compiling a Llama2 model using AOT workflow (`ir=dynamo`)
22+

‎examples/dynamo/torch_compile_resnet_example.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
"""
22
.. _torch_compile_resnet:
33
4-
Compiling ResNet using the Torch-TensorRT `torch.compile` Backend
4+
Compiling ResNet with dynamic shapes using the `torch.compile` backend
55
==========================================================
66
77
This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a ResNet model."""

‎examples/dynamo/torch_compile_stable_diffusion.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
"""
22
.. _torch_compile_stable_diffusion:
33
4-
Torch Compile Stable Diffusion
4+
Compiling Stable Diffusion model using the `torch.compile` backend
55
======================================================
66
77
This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a Stable Diffusion model. A sample output is featured below:

‎examples/dynamo/torch_compile_transformers_example.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
"""
22
.. _torch_compile_transformer:
33
4-
Compiling a Transformer using torch.compile and TensorRT
4+
Compiling BERT using the `torch.compile` backend
55
==============================================================
66
7-
This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a transformer-based model."""
7+
This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a BERT model."""
88

99
# %%
1010
# Imports and Model Definition

‎examples/dynamo/torch_export_gpt2.py

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
"""
22
.. _torch_export_gpt2:
33
4-
Compiling GPT2 using the Torch-TensorRT with dynamo backend
4+
Compiling GPT2 using the dynamo backend
55
==========================================================
66
7-
This interactive script is intended as a sample of the Torch-TensorRT workflow with dynamo backend on a GPT2 model."""
7+
This script illustrates Torch-TensorRT workflow with dynamo backend on popular GPT2 model."""
88

99
# %%
1010
# Imports and Model Definition
@@ -78,9 +78,10 @@
7878
tokenizer.decode(trt_gen_tokens[0], skip_special_tokens=True),
7979
)
8080

81-
# %%
82-
# The output sentences should look like
81+
# Prompt : What is parallel programming ?
82+
8383
# =============================
84-
# Pytorch model generated text: I enjoy walking with my cute dog, but I'm not sure if I'll ever be able to walk with my dog. I'm not sure if I'll ever be able to walk with my
84+
# Pytorch model generated text: The parallel programming paradigm is a set of programming languages that are designed to be used in parallel. The main difference between parallel programming and parallel programming is that
85+
8586
# =============================
86-
# TensorRT model generated text: I enjoy walking with my cute dog, but I'm not sure if I'll ever be able to walk with my dog. I'm not sure if I'll ever be able to walk with my
87+
# TensorRT model generated text: The parallel programming paradigm is a set of programming languages that are designed to be used in parallel. The main difference between parallel programming and parallel programming is that

‎examples/dynamo/torch_export_llama2.py

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
"""
22
.. _torch_export_llama2:
33
4-
Compiling Llama2 using the Torch-TensorRT with dynamo backend
4+
Compiling Llama2 using the dynamo backend
55
==========================================================
66
7-
This interactive script is intended as a sample of the Torch-TensorRT workflow with dynamo backend on a Llama2 model."""
7+
This script illustrates Torch-TensorRT workflow with dynamo backend on popular Llama2 model."""
88

99
# %%
1010
# Imports and Model Definition
@@ -82,9 +82,11 @@
8282
)[0],
8383
)
8484

85-
# %%
86-
# The output sentences should look like
85+
86+
# Prompt : What is dynamic programming?
87+
8788
# =============================
88-
# Pytorch model generated text: I enjoy walking with my cute dog, but I'm not sure if I'll ever be able to walk with my dog. I'm not sure if I'll ever be able to walk with my
89+
# Pytorch model generated text: Dynamic programming is an algorithmic technique used to solve complex problems by breaking them down into smaller subproblems, solving each subproblem only once, and
90+
8991
# =============================
90-
# TensorRT model generated text: I enjoy walking with my cute dog, but I'm not sure if I'll ever be able to walk with my dog. I'm not sure if I'll ever be able to walk with my
92+
# TensorRT model generated text: Dynamic programming is an algorithmic technique used to solve complex problems by breaking them down into smaller subproblems, solving each subproblem only once, and

0 commit comments

Comments
 (0)