Skip to content

Latest commit

 

History

History

deepstream-segmentation

################################################################################
# SPDX-FileCopyrightText: Copyright (c) 2019-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################

Prerequisites:
- DeepStreamSDK 7.0
- Python 3.10
- Gst-python
- NumPy package
- OpenCV package

To install required packages:
$ sudo apt update
$ sudo apt install python3-numpy python3-opencv -y

If on Jetson, the libgomp.so.1 must be added to LD_PRELOAD:
  $ export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1

To run:
  $ python3 deepstream_segmentation.py <config_file> <jpeg/mjpeg stream> <FOLDER NAME TO SAVE FRAMES>

This document shall describe the sample deepstream-segmentation application.

It is meant for simple demonstration of how to use the various DeepStream SDK
elements in the pipeline and extract meaningful insights from a video stream such
as segmentation masks and respective color mapping for segmentation visualization.

This sample creates instance of "nvinfer" element. Instance of
the "nvinfer" uses TensorRT API to execute inferencing on a model. Using a
correct configuration for a nvinfer element instance is therefore very
important as considerable behaviors of the instance are parameterized
through these configs.

For reference, here are the config files used for this sample :
1. The 4-class segmentation model configured through dstest_segmentation_config_semantic.txt 
2. The 2-class segmentation model configured through dstest_segmentation_config_industrial.txt

In this sample, we first create one instance of "nvinfer", referred as the pgie.
for semantic segmentation, it needs semantic model which can get 4 classes map,
including backgroud, car, person, bicycle. Then "nvsegvidsual" plugin chooses 4 different 
colors for them and to display. Similarlty for industrial segmentation, it needs industrial 
model which can only get defective area map. Later nvinfer element attach some MetaData 
to the buffer. By attaching the probe function at the end of the pipeline, one can 
extract meaningful information from this inference. Please refer 
the "tiler_src_pad_buffer_probe" function in the sample code. For details on the 
Metadata format, refer to the file "gstnvdsmeta.h". In this probe we demonstrate
extracting the masks and color mapping for segmentation visualization using opencv 
and numpy.