refactor! : Update README.md with the new library name Torch-TensorRT

Signed-off-by: Dheeraj Peri <[email protected]>

Author: peri044

README.md

@@ -1,10 +1,10 @@
-# TRTorch
+# Torch-TensorRT
 
-[![Documentation](https://img.shields.io/badge/docs-master-brightgreen)](https://nvidia.github.io/TRTorch/)
+[![Documentation](https://img.shields.io/badge/docs-master-brightgreen)](https://nvidia.github.io/Torch-TensorRT/)
 
 > Ahead of Time (AOT) compiling for PyTorch JIT
 
-TRTorch is a compiler for PyTorch/TorchScript, targeting NVIDIA GPUs via NVIDIA's TensorRT Deep Learning Optimizer and Runtime. Unlike PyTorch's Just-In-Time (JIT) compiler, TRTorch is an Ahead-of-Time (AOT) compiler, meaning that before you deploy your TorchScript code, you go through an explicit compile step to convert a standard TorchScript program into an module targeting a TensorRT engine. TRTorch operates as a PyTorch extention and compiles modules that integrate into the JIT runtime seamlessly. After compilation using the optimized graph should feel no different than running a TorchScript module. You also have access to TensorRT's suite of configurations at compile time, so you are able to specify operating precision (FP32/FP16/INT8) and other settings for your module.
+Torch-TensorRT is a compiler for PyTorch/TorchScript, targeting NVIDIA GPUs via NVIDIA's TensorRT Deep Learning Optimizer and Runtime. Unlike PyTorch's Just-In-Time (JIT) compiler, Torch-TensorRT is an Ahead-of-Time (AOT) compiler, meaning that before you deploy your TorchScript code, you go through an explicit compile step to convert a standard TorchScript program into an module targeting a TensorRT engine. Torch-TensorRT operates as a PyTorch extention and compiles modules that integrate into the JIT runtime seamlessly. After compilation using the optimized graph should feel no different than running a TorchScript module. You also have access to TensorRT's suite of configurations at compile time, so you are able to specify operating precision (FP32/FP16/INT8) and other settings for your module.
 
 More Information / System Architecture:
 
@@ -15,18 +15,18 @@ More Information / System Architecture:
 ### C++
 ```c++
 #include "torch/script.h"
-#include "trtorch/trtorch.h"
+#include "torch_tensorrt/torch_tensorrt.h"
 
 ...
 // Set input datatypes. Allowerd options torch::{kFloat, kHalf, kChar, kInt32, kBool}
 // Size of input_dtypes should match number of inputs to the network.
 // If input_dtypes is not set, default precision follows traditional PyT / TRT rules
-auto input = trtorch::CompileSpec::Input(dims, torch::kHalf)
-auto compile_settings = trtorch::CompileSpec({input});
+auto input = torch_tensorrt::CompileSpec::Input(dims, torch::kHalf)
+auto compile_settings = torch_tensorrt::CompileSpec({input});
 // FP16 execution
 compile_settings.enabled_precisions = {torch::kHalf};
 // Compile module
-auto trt_mod = trtorch::CompileGraph(ts_mod, compile_settings);
+auto trt_mod = torch_tensorrt::CompileGraph(ts_mod, compile_settings);
 // Run like normal
 auto results = trt_mod.forward({in_tensor});
 // Save module for later
@@ -36,11 +36,11 @@ trt_mod.save("trt_torchscript_module.ts");
 
 ### Python
 ```py
-import trtorch
+import torch_tensorrt
 
 ...
 compile_settings = {
-    "inputs": [trtorch.Input(
+    "inputs": [torch_tensorrt.Input(
         min_shape=[1, 3, 224, 224],
         opt_shape=[1, 3, 512, 512],
         max_shape=[1, 3, 1024, 1024],
@@ -50,7 +50,7 @@ compile_settings = {
     "enabled_precisions": {torch.half}, # Run with FP16
 }
 
-trt_ts_module = trtorch.compile(torch_script_module, compile_settings)
+trt_ts_module = torch_tensorrt.compile(torch_script_module, compile_settings)
 
 input_data = input_data.half()
 result = trt_ts_module(input_data)
@@ -75,7 +75,7 @@ torch.jit.save(trt_ts_module, "trt_torchscript_module.ts")
 > Note: Refer NVIDIA NGC container(https://ngc.nvidia.com/catalog/containers/nvidia:l4t-pytorch) for PyTorch libraries on JetPack.
 
 ### Dependencies
-These are the following dependencies used to verify the testcases. TRTorch can work with other versions, but the tests are not guaranteed to pass.
+These are the following dependencies used to verify the testcases. Torch-TensorRT can work with other versions, but the tests are not guaranteed to pass.
 
 - Bazel 4.0.0
 - Libtorch 1.9.1 (built with CUDA 11.1)
@@ -85,9 +85,9 @@ These are the following dependencies used to verify the testcases. TRTorch can w
 
 ## Prebuilt Binaries and Wheel files
 
-Releases: https://github.com/NVIDIA/TRTorch/releases
+Releases: https://github.com/NVIDIA/Torch-TensorRT/releases
 
-## Compiling TRTorch
+## Compiling Torch-TensorRT
 
 ### Installing Dependencies
 
@@ -113,17 +113,17 @@ then you have two options.
 
 #### 1. Building using cuDNN & TensorRT tarball distributions
 
-> This is recommended so as to build TRTorch hermetically and insures any bugs are not caused by version issues
+> This is recommended so as to build Torch-TensorRT hermetically and insures any bugs are not caused by version issues
 
-> Make sure when running TRTorch that these versions of the libraries are prioritized in your `$LD_LIBRARY_PATH`
+> Make sure when running Torch-TensorRT that these versions of the libraries are prioritized in your `$LD_LIBRARY_PATH`
 
 1. You need to download the tarball distributions of TensorRT and cuDNN from the NVIDIA website.
     - https://developer.nvidia.com/cudnn
     - https://developer.nvidia.com/tensorrt
 2. Place these files in a directory (the directories `third_party/dist_dir/[x86_64-linux-gnu | aarch64-linux-gnu]` exist for this purpose)
 3. Compile using:
 ``` shell
-bazel build //:libtrtorch --compilation_mode opt --distdir third_party/dist_dir/[x86_64-linux-gnu | aarch64-linux-gnu]
+bazel build //:libtorchtrt --compilation_mode opt --distdir third_party/dist_dir/[x86_64-linux-gnu | aarch64-linux-gnu]
 ```
 
 #### 2. Building using locally installed cuDNN & TensorRT
@@ -170,30 +170,29 @@ new_local_repository(
 ```
 3. Compile using:
 ``` shell
-bazel build //:libtrtorch --compilation_mode opt
+bazel build //:libtorchtrt --compilation_mode opt
 ```
 
 ### Debug build
 ``` shell
-bazel build //:libtrtorch --compilation_mode=dbg
+bazel build //:libtorchtrt --compilation_mode=dbg
 ```
 
 ### Native compilation on NVIDIA Jetson AGX
 ``` shell
-bazel build //:libtrtorch --distdir third_party/dist_dir/aarch64-linux-gnu
+bazel build //:libtorchtrt --distdir third_party/dist_dir/aarch64-linux-gnu
 ```
 > Note: Please refer [installation](docs/tutorials/installation.html) instructions for Pre-requisites
 
 A tarball with the include files and library can then be found in bazel-bin
 
-### Running TRTorch on a JIT Graph
+### Running Torch-TensorRT on a JIT Graph
 
 > Make sure to add LibTorch to your LD_LIBRARY_PATH <br>
->`export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bazel-TRTorch/external/libtorch/lib`
-
+>`export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/bazel-Torch-TensorRT/external/libtorch/lib`
 
 ``` shell
-bazel run //cpp/trtorchexec -- $(realpath <PATH TO GRAPH>) <input-size>
+bazel run //cpp/bin/torchtrtc -- $(realpath <PATH TO GRAPH>) out.ts <input-size>
 ```
 
 ## Compiling the Python Package
@@ -202,15 +201,15 @@ To compile the python package for your local machine, just run `python3 setup.py
 To build wheel files for different python versions, first build the Dockerfile in ``//py`` then run the following
 command
 ```
-docker run -it -v$(pwd)/..:/workspace/TRTorch build_trtorch_wheel /bin/bash /workspace/TRTorch/py/build_whl.sh
+docker run -it -v$(pwd)/..:/workspace/Torch-TensorRT build_torch_tensorrt_wheel /bin/bash /workspace/Torch-TensorRT/py/build_whl.sh
 ```
 Python compilation expects using the tarball based compilation strategy from above.
 
 ## How do I add support for a new op...
 
-### In TRTorch?
+### In Torch-TensorRT?
 
-Thanks for wanting to contribute! There are two main ways to handle supporting a new op. Either you can write a converter for the op from scratch and register it in the NodeConverterRegistry or if you can map the op to a set of ops that already have converters you can write a graph rewrite pass which will replace your new op with an equivalent subgraph of supported ops. Its preferred to use graph rewriting because then we do not need to maintain a large library of op converters. Also do look at the various op support trackers in the [issues](https://github.com/NVIDIA/TRTorch/issues) for information on the support status of various operators.
+Thanks for wanting to contribute! There are two main ways to handle supporting a new op. Either you can write a converter for the op from scratch and register it in the NodeConverterRegistry or if you can map the op to a set of ops that already have converters you can write a graph rewrite pass which will replace your new op with an equivalent subgraph of supported ops. Its preferred to use graph rewriting because then we do not need to maintain a large library of op converters. Also do look at the various op support trackers in the [issues](https://github.com/NVIDIA/Torch-TensorRT/issues) for information on the support status of various operators.
 
 ### In my application?
 
@@ -224,9 +223,9 @@ You can register a converter for your op using the `NodeConverterRegistry` insid
 | ------------- | ------------------------------------------------------------ |
 | [**core**](core)  | Main JIT ingest, lowering, conversion and runtime implementations |
 | [**cpp**](cpp)   | C++ API and CLI source |
-| [**examples**](examples)   | Example applications to show different features of TRTorch |
-| [**py**](py)   | Python API for TRTorch |
-| [**tests**](tests) | Unit tests for TRTorch |
+| [**examples**](examples)   | Example applications to show different features of Torch-TensorRT |
+| [**py**](py)   | Python API for Torch-TensorRT |
+| [**tests**](tests) | Unit tests for Torch-TensorRT |
 
 ## Contributing
 
@@ -235,4 +234,4 @@ Take a look at the [CONTRIBUTING.md](CONTRIBUTING.md)
 
 ## License
 
-The TRTorch license can be found in the LICENSE file. It is licensed with a BSD Style licence
+The Torch-TensorRT license can be found in the LICENSE file. It is licensed with a BSD Style licence

core/README.md

@@ -1,9 +1,9 @@
-# TRTorch Core
-The TRTorch Core is the main graph analysis library, it processes a TorchScript Module, converting method graphs to engines and returning a new equivalent module which when run will run inputs through a TensorRT engine
+# Torch-TensorRT Core
+The Torch-TensorRT Core is the main graph analysis library, it processes a TorchScript Module, converting method graphs to engines and returning a new equivalent module which when run will run inputs through a TensorRT engine
 
 ## Stages
 
-> Basic rule of thumb for organization, if the the output of the component is a modified block then it is in lowering, if the output is a TRT engine block then its in conversion 
+> Basic rule of thumb for organization, if the the output of the component is a modified block then it is in lowering, if the output is a TRT engine block then its in conversion
 
 ## Lowering Passes
 
@@ -14,29 +14,29 @@ Firstly the graph will go through the lowering passes used in LibTorch, this wil
 
 #### Call Method Insertions
 
-Graphs from prim::CallMethods need to be inserted into the graph or used to segment the graph into convertible subgraphs. 
+Graphs from prim::CallMethods need to be inserted into the graph or used to segment the graph into convertible subgraphs.
 
-### TRTorch Lowering
+### Torch-TensorRT Lowering
 
-To simplify conversion we can use the PyTorch JIT Subgraph Rewriter to simplify the set of subgraphs that need explicit TensorRT converters. This means we could aim for closer to 1->1 op conversion vs looking for applicable subgraphs, limit the number of converters and reduce the size of each converter. 
+To simplify conversion we can use the PyTorch JIT Subgraph Rewriter to simplify the set of subgraphs that need explicit TensorRT converters. This means we could aim for closer to 1->1 op conversion vs looking for applicable subgraphs, limit the number of converters and reduce the size of each converter.
 
 
-## Conversion Phase 
+## Conversion Phase
 
-Once the graph has be simplified to a form thats easy to convert, we then set up a conversion context to manage the construction of a TensorRT INetworkDefinition from the blocks nodes. The conversion context records the set of converted nodes, block inputs and outputs and other information about the conversion of the graph. This data is then used to help converters link together layers and also hold build time information like weights required to construct the engine. After the context is created, the block converter starts iterating through the list of nodes, for each node, the converter will look at its inputs and assemble a dictionary of resources to pass to the converter. Inputs can be in a couple of states: 
+Once the graph has be simplified to a form thats easy to convert, we then set up a conversion context to manage the construction of a TensorRT INetworkDefinition from the blocks nodes. The conversion context records the set of converted nodes, block inputs and outputs and other information about the conversion of the graph. This data is then used to help converters link together layers and also hold build time information like weights required to construct the engine. After the context is created, the block converter starts iterating through the list of nodes, for each node, the converter will look at its inputs and assemble a dictionary of resources to pass to the converter. Inputs can be in a couple of states:
 - The input is a block parameter
   In this case the input should have already been stored in as an IValue in the conversion context evaluated_value_map. The conversion stage will add the IValue to the list of args for the converter
 - The input is an output of a node that has already been converted
   In this case the ITensor of the output has added to the to the value_tensor_map, The conversion stage will add the ITensor to the list of args for the converter
 - The input is from a node that produces a static value
   There are nodes that produce static values, typically used to store parameters for operators, we need to evaluate these nodes at conversion time to be able to convert a op. The converter will look for a node evaluator in the evaluator registry and run it on the node. The IValue produced will be entered in the conversion context evaluated_value_map and added to the list of args for the converter.
-- The input is from a node that has not been converted 
-  TRTorch will error here 
+- The input is from a node that has not been converted
+  Torch-TensorRT will error here
 
 ### Node Evaluation
 
-There are some nodes that contain static data and are resources for operations. These can be evaluated at conversion time so that you can use those values when doing node conversion. In theory any node kind can have a conversion time evaluator as long as it produces a static IValue, This IValue will be stored in the conversion context so it can be consumed by any node that takes the evaluated node as an input. 
+There are some nodes that contain static data and are resources for operations. These can be evaluated at conversion time so that you can use those values when doing node conversion. In theory any node kind can have a conversion time evaluator as long as it produces a static IValue, This IValue will be stored in the conversion context so it can be consumed by any node that takes the evaluated node as an input.
 
 ### Converters
 
-See the README in //core/conversion/converters for more information
+See the README in //core/conversion/converters for more information

core/conversion/evaluators/README.md

@@ -1,2 +1,6 @@
-Three types 
-bool, int64_t, double
+# Evaluators
+
+   Operators whose outputs are known at compile time are considered to be evaluators. For example, in PyTorch library,
+   operations like `aten::zeros`, `aten::full` are constants in the graph which can be expressed as constant values in TensorRT
+   graph. Pytorch library expresses constants and new data structures using `aten` and `prim` libraries. Corresponding TensorRT
+   implementations for such operators can be found in `aten.cpp` and `prim.cpp`.

core/plugins/README.md

@@ -1,19 +1,19 @@
-# TRTorch Plugins
+# Torch-TensorRT Plugins
 
-A library for plugins (custom layers) used in a network. This component of TRTorch library builds a separate library called `libtrtorch_plugins.so`.
+A library for plugins (custom layers) used in a network. This component of Torch-TensorRT library builds a separate library called `libtorchtrt_plugins.so`.
 
-On a high level, TRTorch plugin library interface does the following :
+On a high level, Torch-TensorRT plugin library interface does the following :
 
 - Uses TensorRT plugin registry as the main data structure to access all plugins.
 
 - Automatically registers TensorRT plugins with empty namepsace.
 
-- Automatically registers TRTorch plugins with `"trtorch"` namespace.
+- Automatically registers Torch-TensorRT plugins with `"torch_tensorrt"` namespace.
 
 Here is the brief description of functionalities of each file
 
-- `plugins.h` - Provides a macro to register any plugins with `"trtorch"`  namespace.
-- `register_plugins.cpp` - Main registry class which initializes both `libnvinfer` plugins and TRTorch plugins (`Interpolate` and `Normalize`)
+- `plugins.h` - Provides a macro to register any plugins with `"torch_tensorrt"`  namespace.
+- `register_plugins.cpp` - Main registry class which initializes both `libnvinfer` plugins and Torch-TensorRT plugins (`Interpolate` and `Normalize`)
 - `impl/interpolate_plugin.cpp` - Core implementation of interpolate plugin. Uses pytorch kernels during execution.
 - `impl/normalize_plugin.cpp` - Core implementation of normalize plugin. Uses pytorch kernels during execution.
 
@@ -22,19 +22,19 @@ A converter basically converts a pytorch layer in the torchscript graph into a T
 We can access a plugin via the plugin name and namespace in which it is registered.
 For example, to access the Interpolate plugin, we can use
 ```
-auto creator = getPluginRegistry()->getPluginCreator("Interpolate", "1", "trtorch");
+auto creator = getPluginRegistry()->getPluginCreator("Interpolate", "1", "torch_tensorrt");
 auto interpolate_plugin = creator->createPlugin(name, &fc); // fc is the collection of parameters passed to the plugin.
 ```
 
 ### If you have your own plugin
 
-If you'd like to compile your plugin with TRTorch,
+If you'd like to compile your plugin with Torch-TensorRT,
 
 - Add your implementation to the `impl` directory
-- Add a call `REGISTER_TRTORCH_PLUGINS(MyPluginCreator)`  to `register_plugins.cpp`. `MyPluginCreator` is the plugin creator class which creates your plugin. By adding this to `register_plugins.cpp`, your plugin will be initialized and accessible (added to TensorRT plugin registry) during the `libtrtorch_plugins.so` library loading.
+- Add a call `REGISTER_TRTORCH_PLUGINS(MyPluginCreator)`  to `register_plugins.cpp`. `MyPluginCreator` is the plugin creator class which creates your plugin. By adding this to `register_plugins.cpp`, your plugin will be initialized and accessible (added to TensorRT plugin registry) during the `libtorchtrt_plugins.so` library loading.
 - Update the `BUILD` file with the your plugin files and dependencies.
 - Implement a converter op which makes use of your plugin.
 
-Once you've completed the above steps, upon successful compilation of TRTorch library, your plugin should be available in  `libtrtorch_plugins.so`.
+Once you've completed the above steps, upon successful compilation of Torch-TensorRT library, your plugin should be available in  `libtorchtrt_plugins.so`.
 
-A sample runtime application on how to run a network with plugins can be found <a href="https://github.com/NVIDIA/TRTorch/tree/master/examples/trtorchrt_example" >here</a>
+A sample runtime application on how to run a network with plugins can be found <a href="https://github.com/NVIDIA/Torch-TensorRT/tree/master/examples/trtorchrt_example" >here</a>