Update for DeepStream 7.1

Author: fc-camel

.gitattributes

@@ -0,0 +1,4 @@
+deepstream_parallel_inference_app/tritonserver/models/bodypose2d/1/model.onnx filter=lfs diff=lfs merge=lfs -text
+deepstream_parallel_inference_app/tritonserver/models/yolov4/1/yolov4_-1_3_416_416_dynamic.onnx.nms.onnx filter=lfs diff=lfs merge=lfs -text
+legacy_apps/deepstream-retail-analytics/files/0001_compressed.h264 filter=lfs diff=lfs merge=lfs -text
+legacy_apps/deepstream-retail-analytics/files/basketClassifier.etlt filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,6 +1,6 @@
-# Reference Apps using DeepStream 7.0
+# Reference Apps using DeepStream 7.1
 
-This repository contains the reference applications for video analytics tasks using TensorRT and DeepSTream SDK 7.0.
+This repository contains the reference applications for video analytics tasks using TensorRT and DeepSTream SDK 7.1.
 
 ## Getting Started ##
 We currently provide three different reference applications:
@@ -12,20 +12,20 @@ To clone the project in the above directory, sudo permission may be required.
 
 For further details, please see each project's README.
 
-### back-to-back-detectors : [README](back-to-back-detectors/README.md) ###
-  The project shows usage of 2 detectors in cascaded mode
-  ![sample back-to-back-detectors output](back-to-back-detectors/.backtobackdetectors.png)
 ### Anomaly Detection : [README](anomaly/README.md) ###
   The project contains auxiliary dsdirection plugin to show the capability of DeepstreamSDK in anomaly detection.
   ![sample anomaly output](anomaly/.opticalflow.png)
 ### Runtime Source Addition Deletion: [README](runtime_source_add_delete/README.md) ###
   The project demonstrates addition and deletion of video sources in a live Deepstream pipeline.
-### Body Pose 3D: [README](deepstream-bodypose-3d/README.md) ###
-  The project demonstrates usage of Body Pose 3D model in Deepstream application
-  ![sample anomaly output](deepstream-bodypose-3d/sources/.screenshot.png)
 ### Single-View 3D Tracking: [README](deepstream-tracker-3d/README.md) ###
   The sample demostrates usage of single-view 3D tracking with DeepStream nvtracker.
   ![sample 3D tracking output](deepstream-tracker-3d/figures/.retail_viz.png)
-### Retail Analytics: [README](deepstream-retail-analytics/README.md) ###
-  The project shows real-time Intelligent Video Analytics (IVA) in a brick and mortar retail environment.
+### Parallel Multiple Models Inferencing: [README](deepstream_parallel_inference_app/README.md) ###
+  The project demonstrate how to implement multiple models inferencing in parallel with DeepStream APIs.
+### Bodypose 3D Model Inferencing: [README](deepstream-bodypose-3d/README.md) ###
+  The sample demonstrate how to customize the multiple input layers model preprocessing and the customization of the bodypose 3D model postprocessing.
+### Redaction with DeepStream sample: [README](redaction_with_deepstream/README.md) ###
+  This sample demonstrate how to obscure the designated type of objects in the video.
 
+## Legacy DeepStream Reference Samples
+Some old samples are not supported any more for different reasons. The legacy samples are moved to the legacy folder /legacy_apps.

anomaly/.dsdirection_pipeline.png

@@ -1,4 +1,4 @@
-# ANOMALY DETECTION REFERENCE APP USING DEEPSTREAMSDK 7.0
+# ANOMALY DETECTION REFERENCE APP USING DEEPSTREAMSDK 7.1
 
 ## Introduction
 The project contains anomaly detection application and auxiliary plug-ins to show the
@@ -58,7 +58,7 @@ Anomaly detection app pipeline:
  $ cd sources/apps/sample_apps/deepstream_reference_apps/anomaly/
  $ cd apps/deepstream-anomaly-detection-test/
  $ Set CUDA_VER in the MakeFile as per platform.
-     For both x86 & Jetson, CUDA_VER=12.2
+     For both x86 & Jetson, CUDA_VER=12.6
  $ sudo make
 
  $ ./deepstream-anomaly-detection-app <uri1> [uri2] ... [uriN]

anomaly/.opticalflow.png

@@ -1,11 +1,12 @@
 ################################################################################
-# Copyright (c) 2020-2021, NVIDIA CORPORATION.  All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2020-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
+# http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,

anomaly/README.md

@@ -1,11 +1,12 @@
 /*
- * Copyright (c) 2020-2024, NVIDIA CORPORATION.  All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2020-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
  * You may obtain a copy of the License at
  *
- *     http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,

anomaly/apps/deepstream-anomaly-detection-test/Makefile

@@ -1,11 +1,12 @@
 ################################################################################
-# Copyright (c) 2019-2023, NVIDIA CORPORATION.  All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2019-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
+# http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
@@ -15,9 +16,6 @@
 ################################################################################
 
 # Following properties are mandatory when engine files are not specified:
-#   int8-calib-file(Only in INT8)
-#   Caffemodel mandatory properties: model-file, proto-file, output-blob-names
-#   UFF: uff-file, input-dims, uff-input-blob-name, output-blob-names
 #   ONNX: onnx-file
 #
 # Mandatory properties for detectors:
@@ -54,23 +52,17 @@
 [property]
 gpu-id=0
 net-scale-factor=0.00392156862745098
-tlt-model-key=tlt_encode
-tlt-encoded-model=../../../../../../../samples/models/Primary_Detector/resnet18_trafficcamnet.etlt
-model-engine-file=../../../../../../../samples/models/Primary_Detector/resnet18_trafficcamnet.etlt_b30_gpu0_int8.engine
+onnx-file=../../../../../../../samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx
+model-engine-file=../../../../../../../samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx_b1_gpu0_int8.engine
 labelfile-path=../../../../../../../samples/models/Primary_Detector/labels.txt
 int8-calib-file=../../../../../../../samples/models/Primary_Detector/cal_trt.bin
-force-implicit-batch-dim=1
 batch-size=1
 process-mode=1
 model-color-format=0
 network-mode=1
 num-detected-classes=4
 interval=0
 gie-unique-id=1
-uff-input-order=0
-uff-input-blob-name=input_1
-output-blob-names=output_cov/Sigmoid;output_bbox/BiasAdd
-infer-dims=3;544;960
 cluster-mode=2
 
 [class-attrs-all]

anomaly/apps/deepstream-anomaly-detection-test/deepstream_anomaly_detection_test.c

@@ -1,36 +1,30 @@
 ################################################################################
-# SPDX-FileCopyrightText: Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
-# SPDX-License-Identifier: MIT
+# SPDX-FileCopyrightText: Copyright (c) 2023-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
 #
-# Permission is hereby granted, free of charge, to any person obtaining a
-# copy of this software and associated documentation files (the "Software"),
-# to deal in the Software without restriction, including without limitation
-# the rights to use, copy, modify, merge, publish, distribute, sublicense,
-# and/or sell copies of the Software, and to permit persons to whom the
-# Software is furnished to do so, subject to the following conditions:
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
 #
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
+# http://www.apache.org/licenses/LICENSE-2.0
 #
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-# DEALINGS IN THE SOFTWARE.
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
 ################################################################################
 infer_config {
   unique_id: 1
   gpu_ids: [0]
   max_batch_size: 30
   backend {
     inputs: [ {
-      name: "input_1"
+      name: "input_1:0"
     }]
     outputs: [
-      {name: "output_cov/Sigmoid"},
-      {name: "output_bbox/BiasAdd"}
+      {name: "output_cov/Sigmoid:0"},
+      {name: "output_bbox/BiasAdd:0"}
     ]
     triton {
       model_name: "Primary_Detector"
@@ -44,7 +38,7 @@ infer_config {
   preprocess {
     network_format: MEDIA_FORMAT_NONE
     tensor_order: TENSOR_ORDER_LINEAR
-    tensor_name: "input_1"
+    tensor_name: "input_1:0"
     maintain_aspect_ratio: 0
     frame_scaling_hw: FRAME_SCALING_HW_GPU
     frame_scaling_filter: 1

anomaly/apps/deepstream-anomaly-detection-test/dsanomaly_pgie_config.txt

@@ -1,11 +1,12 @@
 ################################################################################
-# Copyright (c) 2019-2021, NVIDIA CORPORATION.  All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2019-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
+# http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,

anomaly/apps/deepstream-anomaly-detection-test/dsanomaly_pgie_nvinferserver_config.txt

@@ -1,11 +1,12 @@
 ################################################################################
-# Copyright (c) 2019-2020, NVIDIA CORPORATION.  All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2019-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
+# http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,

anomaly/plugins/gst-dsdirection/Makefile

@@ -1,11 +1,12 @@
 /*
- * Copyright (c) 2019-2020, NVIDIA CORPORATION.  All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2019-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
  * You may obtain a copy of the License at
  *
- *     http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,

anomaly/plugins/gst-dsdirection/dsdirection_lib/Makefile

@@ -1,11 +1,12 @@
 /*
- * Copyright (c) 2019-2020, NVIDIA CORPORATION.  All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2019-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
  * You may obtain a copy of the License at
  *
- *     http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,

anomaly/plugins/gst-dsdirection/dsdirection_lib/dsdirection_lib.cpp

@@ -1,11 +1,12 @@
 /*
- * Copyright (c) 2019-2020, NVIDIA CORPORATION.  All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2019-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
  * You may obtain a copy of the License at
  *
- *     http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,

anomaly/plugins/gst-dsdirection/dsdirection_lib/dsdirection_lib.h

@@ -1,11 +1,12 @@
 /*
- * Copyright (c) 2019-2020, NVIDIA CORPORATION.  All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2019-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
  * You may obtain a copy of the License at
  *
- *     http://www.apache.org/licenses/LICENSE-2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,

anomaly/plugins/gst-dsdirection/gstdsdirection.cpp

@@ -1,23 +1,16 @@
-MIT License
+SPDX-FileCopyrightText: Copyright (c) 2018-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-License-Identifier: Apache-2.0
 
-Copyright (c) 2021 NVIDIA CORPORATION. All rights reserved.
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
 
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
+http://www.apache.org/licenses/LICENSE-2.0
 
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
 
 

anomaly/plugins/gst-dsdirection/gstdsdirection.h

@@ -6,50 +6,27 @@ The project contains 3D Body Pose application built using  Deepstream SDK.
 This application is built for [KAMA: 3D Keypoint Aware Body Mesh Articulation](https://arxiv.org/abs/2104.13502).
 ![sample pose output](./sources/.screenshot.png)
 ## Prerequisites:
-DeepStream SDK 7.0 installed which is available at  http://developer.nvidia.com/deepstream-sdk
+DeepStream SDK 7.1 installed which is available at  http://developer.nvidia.com/deepstream-sdk
 Please follow instructions in the `/opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-app/README` on how
 to install the prequisites for building Deepstream SDK apps.
 
+The pretrained TAO models [PeopleNet](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplenet) and [BodyPose3DNet](https://ngc.nvidia.com/models/nvstaging:tao:bodypose3dnet) from NGC.
+
 ## Installation
 Follow https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Quickstart.html to setup the DeepStream SDK
 
 1. Preferably clone the app in
   `/opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/`
-and define project home as `export BODYPOSE3D_HOME=<parent-path>/deepstream-bodypose-3d`.
+and define project home as `export BODYPOSE3D_HOME=<parent-path>/3d-bodypose-deepstream`.
 
-2. Install [NGC CLI](https://ngc.nvidia.com/setup/installers/cli) and download [PeopleNet](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/peoplenet) and [BodyPose3DNet](https://ngc.nvidia.com/models/nvstaging:tao:bodypose3dnet) from NGC.
-```bash
-$ mkdir -p $BODYPOSE3D_HOME/models
-$ cd $BODYPOSE3D_HOME/models
-# Download PeopleNet
-$ ngc registry model download-version "nvidia/tao/peoplenet:deployable_quantized_v2.5"
-# Download BodyPose3DNet
-$ ngc registry model download-version "nvidia/tao/bodypose3dnet:deployable_accuracy_v1.0"
-```
-
-By now the directory tree should look like this
-```bash
-$ tree $BODYPOSE3D_HOME -d
-$BODYPOSE3D_HOME
-├── configs
-├── models
-│   ├── bodypose3dnet_vdeployable_accuracy_v1.0
-│   └── peoplenet_vdeployable_quantized_v2.5
-├── sources
-│   ├── deepstream-sdk
-│   └── nvdsinfer_custom_impl_BodyPose3DNet
-└── streams
+2. Install Eigen development packages
 ```
-
-3. Download and extract [Eigen 3.4.0](https://eigen.tuxfamily.org/index.php?title=Main_Page) under the project foler.
-```bash
-$ cd $BODYPOSE3D_HOME
-$ wget https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz
-$ tar xvzf eigen-3.4.0.tar.gz
-$ ln eigen-3.4.0 eigen -s
+  sudo apt install libeigen3-dev
+  cd /usr/include
+  sudo ln -sf eigen3/Eigen Eigen
 ```
 
-4. For Deepstream SDK version older than 6.2, copy and build custom `NvDsEventMsgMeta` into Deepstream SDK installation path. Copy and build custom `NvDsEventMsgMeta` into Deepstream SDK installation path.
+3. For Deepstream SDK version older than 6.2, copy and build custom `NvDsEventMsgMeta` into Deepstream SDK installation path. Copy and build custom `NvDsEventMsgMeta` into Deepstream SDK installation path.
 The custom `NvDsEventMsgMeta` structure handles pose3d and pose25d meta data.
 ```bash
 # Copy deepstream sources
@@ -73,6 +50,12 @@ If the above steps are successful, `deepstream-pose-estimation-app` shall be bui
 
 ## Run the applications
 ### `deepstream-pose-estimation-app`
+Download the pretrained TAO models with the download script
+```
+cd $BODYPOSE3D_HOME
+bash ./download_models.sh
+```
+
 The command line options of this application are listed below:
 ```bash
 $ ./deepstream-pose-estimation-app -h