Configuring The App
It was so hard for me to understand what was going on when I first touched the DeepStream SDK, so for this post, I want to share how the DeepStream App is configured VISUALLY. (Hoping I’m understanding it correctly)
Required Configuration File
You need 2 configuration files for the deepstream-app.
1. Deepstream-app.txt
2. config_infer.txt
1.Deepstream-app.txt
For this file, You’ll need to assign all the settings for the pipeline itself. What kind of input should the pipeline be expecting? Is it a live camera? or an mp4 video? Do you want a video as an output?
Also, you’ll need to assign a 2.config_infer.txt file for configuring the inference settings.
2. config_infer.txt
For this file, you’ll need to assign all the information that is inference-related.
2-1. Which Engine or model file should the pipeline use?
2-2. What about the label data?
2-3. How do you want to post-process?
So at this point, in total, you need 5(2 files to configure the pipeline and 3 files that are required in config_infer.txt) files.
※When you run the app, your directory would look something like this
~/Directory-for-deepstream-app/├── deepstream_app.txt ├── config_infer.txt ├── post_process_parser.so ├── model_data.uff OR model.engine #File for the model: File-extentions depends on the framework for your model └── labels_data.txt
Gathering Necessary Files Required in config_infer.txt
In the last step, I’ve told you we’ll be needing 3 files for config_infer.txt like the diagram below.
– Label Data
– Post-Processing Parser
– TensorRT Engine
Label Data
This is required when creating the model so it should already exist by this point.
Post-Processing Parser
This is where you write the code in C++ and build it, but if you haven’t written c++ before, it might be better to use parsers that someone has already made. I want you to check the link for further descriptions, but basically what this file is doing here is, manually extracting the coordinates and confidence score for the bounding boxes after the inference is done in the deepstream-app.
Check here.
TensorRT Engine
The DeepStream App uses a tensorRT engine file for inference, so there is a need to convert models to engine files depending on which framework it was built. (If you are using a triton server, you won’t have to do this process but you’ll need additional configuration files. By the way, using a triton server is recommended. I’m not using(more like I can’t use) it due to specific needs to use older versions of deepstream)
In order to understand how to get/assign the engine file to the pipeline, you’ll have to first understand the workflow when the app starts(Diagram below).
Considering the workflow from above, there are two ways to get/assign the engine file.
1. Convert your model to an engine file, then assign that in deepstream-app.txt OR config_infer.txt.
2. Convert your model to a file that the app can convert to an engine file during execution, then assign that in deepstream-app.txt OR config_infer.txt.
I’m mainly working on option 2, and the routes I was able to find were like the diagram below.
Verdicts And Thoughts
Now you know INTUITIVELY, how to get all the required files.
It took me A LOT of time to get to this point, BUT, I’m really amazed about how deepstream simplified this pipeline-creating process. If I can understand more about the models themselves(I still don’t know exactly what is going on), I think I’ll be able to create these kinds of pipelines at a super-fast pace.
I’m hoping I can get to that Level.
