“Modernizing” ConvNets “ConvNext” is a gradually “modernized” traditional ConvNet model designed to reexamine the design spaces and test the limits of what a pure ConvNet can achieve. The paper does this by modifying the Micro Design of the ConvNet architecture…
Batch Normalization Batch Normalization is a milestone technique in the development of deep learning, enabling various networks to train. However, BN’s error increases rapidly when the batch size becomes small affecting the batch statistics estimation. Furthermore, the concept of “batch”…
Abstract In existing Transformer-based models, tokens are all of a fixed scale, a property unsuitable for vision applications such as semantic segmentation that require dense prediction at the pixel level. In addition, due to the computational complexity of its self-attention…
Structure Gaussian Error Unit is a high-performing neural network activation function that weights inputs by their value, rather than gates inputs by their sign as in ReLUs. GELU is defined as the equation in the image. Results GELU exceeds the…
I finished reading “Barking Up The Wrong Tree” by Eric Barker , so I like to share my top 3 messages from this book. Top 3 Key Takeaways The alternative to self-confidence is Self-compassion; You must not fool yourself and you…
What is it? Confound variables are like extra independent variables that affect the results. Issues This can cause the following issues. Increase Variance Introduce Bias Avoidance Here are some methods to avoid the above. Control considering variables Random assignment Counterbalancing
Creating Templates Jinja2 is a template engine for Python which can be useful when you want to keep track of model training configuration versions. Here is one way to Implement it by combining them with a YAML file. 1. Create…
Abstract Point Pillar is an architecture proposed for 3D object detection using point clouds as inputs. Architecture The architecture consists of mainly 3 elements. Pillar Feature Net BackBone Detection Head 1. Pillar Feature Net This phase takes the following steps.…
Depth Estimation The main challenges in monocular 3D object detection lie in accurately predicting object depth. CaDDN(Categorical Depth Distribution Network) uses a predicted categorical depth distribution for each pixel to project appropriate depth in 3D space. Approaches There are several…
The 2 Stages There are mainly 2 stages when training a generative model. 1. Perceptual Compression Process of removing high-frequency details Encapsulate data into an abstract representation GANS accomplish this by projecting data from pixel space to a hyperspace called…