Batch Normalization Batch Normalization is a milestone technique in the development of deep learning, enabling various networks to train. However, BN’s error increases rapidly when the batch size becomes small affecting the batch statistics estimation. Furthermore, the concept of “batch”…
Abstract In existing Transformer-based models, tokens are all of a fixed scale, a property unsuitable for vision applications such as semantic segmentation that require dense prediction at the pixel level. In addition, due to the computational complexity of its self-attention…
Structure Gaussian Error Unit is a high-performing neural network activation function that weights inputs by their value, rather than gates inputs by their sign as in ReLUs. GELU is defined as the equation in the image. Results GELU exceeds the…
What is it? Confound variables are like extra independent variables that affect the results. Issues This can cause the following issues. Increase Variance Introduce Bias Avoidance Here are some methods to avoid the above. Control considering variables Random assignment Counterbalancing
Creating Templates Jinja2 is a template engine for Python which can be useful when you want to keep track of model training configuration versions. Here is one way to Implement it by combining them with a YAML file. 1. Create…
Abstract Point Pillar is an architecture proposed for 3D object detection using point clouds as inputs. Architecture The architecture consists of mainly 3 elements. Pillar Feature Net BackBone Detection Head 1. Pillar Feature Net This phase takes the following steps.…
Depth Estimation The main challenges in monocular 3D object detection lie in accurately predicting object depth. CaDDN(Categorical Depth Distribution Network) uses a predicted categorical depth distribution for each pixel to project appropriate depth in 3D space. Approaches There are several…
The 2 Stages There are mainly 2 stages when training a generative model. 1. Perceptual Compression Process of removing high-frequency details Encapsulate data into an abstract representation GANS accomplish this by projecting data from pixel space to a hyperspace called…
Pytorch Profiler can help you detect performance bottlenecks when training/deploying a model Here is one implementation import torch import torchvision.models as models from torch.profiler import profile, record_function, ProfilerActivity model = models.resnet18() inputs = torch.randn(1, 3, 512, 512) with profile(activities=[ProfilerActivity.CPU], record_shapes=True)…
Types Here are some approaches to measure depth by only using RGB images from motion. Optical Expansion Observe how the length of an object changes as the camera moves closer. If the object is close, the length will dramatically change…