124. Preprocessing for Deepstream

I found out why my TensorRT engine model was not working as expected. I messed up with configuring the preprocessing step for Deepstream.

When you use Deepstream to run inference, there is a property called net-scale-factor and offsets which you need to configure depending on your preprocessing methods(The methods you used for training).

The preprocessing method Deepstream is going to apply is defined in this equation.

#x: Input 
#y: Input After applying preprocessing
y = net-scale-factor*(x - offsets)

You can configure the 2 elements inside the config file for gst-nvinfer like this..

[property]
...
net-scale-factor=1
offsets=offsets_for_R;offsets_for_G;offsets_for_B;
...
# note that you can set offsets for each channel individually

Let’s say you want to use a pytorch model.
A typical pytorch model requires you to…
1. Apply min-max normalization
2. Standardize it using the imagenet’s mean and standard deviation(mean=[0.485, 0.456, 0.406],std=[0.229, 0.224, 0.225]).

So, in step 1, you are dividing the input with the max value,which is 255.

x/255

Finally, in step 2, you are standardizing the input.

#x: Input 
#y: Input After applying preprocessing
#μ: Imagenet mean=[0.485, 0.456, 0.406]
#σ: Imagenet standard deviation=[0.229, 0.224, 0.225]
y = (x/255-μ)/σ

Let’s reshape this to make it look similar to the equation.

y= (x - μ*255)/σ*255

Now, if you compare this to the equation above you can tell that…

offsets = μ*255
net-scale-factor = 1/σ*255
  • For offsets,you can assign it for each channel(μ=[R,G,B]=[0.485, 0.456, 0.406])
    So, the final offsets values are going to be the following.

    offsets=offsets_for_R;offsets_for_G;offsets_for_B;
offsets=255*0.485;255*0.456;255*0.406;
  • For net-scale-factor, you can only assign 1, so you calculate the average of std for 3 channels(Since the values are really small),and use that. The final net-scale-factor values are going to be the following. 
    σAVE=(0.229+0.224+0.225)/3
net-scale-factor=1/σAVE*255

Related Posts