1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
|
- ################################################################################
- # Copyright (c) 2020-2021, NVIDIA CORPORATION. All rights reserved.
- #
- # Permission is hereby granted, free of charge, to any person obtaining a
- # copy of this software and associated documentation files (the "Software"),
- # to deal in the Software without restriction, including without limitation
- # the rights to use, copy, modify, merge, publish, distribute, sublicense,
- # and/or sell copies of the Software, and to permit persons to whom the
- # Software is furnished to do so, subject to the following conditions:
- #
- # The above copyright notice and this permission notice shall be included in
- # all copies or substantial portions of the Software.
- #
- # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- # THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
- # DEALINGS IN THE SOFTWARE.
- ################################################################################
- Prerequisites:
- - DeepStreamSDK 5.1
- - Python 3.6
- - Gst-python
- - NumPy package
- - OpenCV package
- To install required packages:
- $ sudo apt update
- $ sudo apt install python3-numpy python3-opencv -y
- To run:
- $ python3 deepstream_imagedata-multistream.py <uri1> [uri2] ... [uriN] <FOLDER NAME TO SAVE FRAMES>
- e.g.
- $ python3 deepstream_imagedata-multistream.py file:///home/ubuntu/video1.mp4 file:///home/ubuntu/video2.mp4 frames
- $ python3 deepstream_imagedata-multistream.py rtsp://127.0.0.1/video1 rtsp://127.0.0.1/video2 frames
- This document describes the sample deepstream-imagedata-multistream application.
- This sample builds on top of the deepstream-test3 sample to demonstrate how to:
- * Access imagedata in a multistream source
- * Modify the images in-place. Changes made to the buffer will reflect in the downstream but
- color format, resolution and numpy transpose operations are not permitted.
- * Make a copy of the image, modify it and save to a file. These changes are made on the copy
- of the image and will not be seen downstream.
- * Extract the stream metadata, imagedata, which contains useful information about the
- frames in the batched buffer.
- * Annotating detected objects within certain confidence interval
- * Use OpenCV to draw bboxes on the image and save it to file.
- * Use multiple sources in the pipeline.
- * Use a uridecodebin so that any type of input (e.g. RTSP/File), any GStreamer
- supported container format, and any codec can be used as input.
- * Configure the stream-muxer to generate a batch of frames and infer on the
- batch for better resource utilization.
- NOTE:
- - For x86, only CUDA unified memory is supported.
- - Only RGBA color format is supported for access from Python. Color conversion
- is added in the pipeline for this reason.
- This sample accepts one or more H.264/H.265 video streams as input. It creates
- a source bin for each input and connects the bins to an instance of the
- "nvstreammux" element, which forms the batch of frames. The batch of
- frames is fed to "nvinfer" for batched inferencing. The batched buffer is
- composited into a 2D tile array using "nvmultistreamtiler." The rest of the
- pipeline is similar to the deepstream-test3 and deepstream-imagedata sample.
- The "width" and "height" properties must be set on the stream-muxer to set the
- output resolution. If the input frame resolution is different from
- stream-muxer's "width" and "height", the input frame will be scaled to muxer's
- output resolution.
- The stream-muxer waits for a user-defined timeout before forming the batch. The
- timeout is set using the "batched-push-timeout" property. If the complete batch
- is formed before the timeout is reached, the batch is pushed to the downstream
- element. If the timeout is reached before the complete batch can be formed
- (which can happen in case of rtsp sources), the batch is formed from the
- available input buffers and pushed. Ideally, the timeout of the stream-muxer
- should be set based on the framerate of the fastest source. It can also be set
- to -1 to make the stream-muxer wait infinitely.
- The "nvmultistreamtiler" composite streams based on their stream-ids in
- row-major order (starting from stream 0, left to right across the top row, then
- across the next row, etc.).
|