▮ Execution Methods Execution methods can become confusing to people who have just started machine learning. So for this post, I’d like to share the difference between graph execution and eager execution. ▮ Graph Execution Fig.1 – Graph Execution Graph…
When applying data augmentation for computer vision models, it is important to apply the same augmentation in the same order for both the input and target image. Here is one way to implement using Albumentation and Pytorch. Implementation Today I’ll…
Implementation Saving checkpoints for model weights during training can be helpful in the case of the following examples. Want to resume training later Avoid losing weight data when the process stops during training due to some kind of error. Restore…
Pytorch Profiler can help you detect performance bottlenecks when training/deploying a model Here is one implementation import torch import torchvision.models as models from torch.profiler import profile, record_function, ProfilerActivity model = models.resnet18() inputs = torch.randn(1, 3, 512, 512) with profile(activities=[ProfilerActivity.CPU], record_shapes=True)…
Running Pytorch without Python TorchScript enables users to load Pytorch models in processes where there is no python dependency. Instead of running the process in Python runtime, it converts the model to be able to run in an independent “Torchsript”…
Deploying Your Model TorchServe allows you to expose a WEB API for your Pytorch model that may be accessed directly or via your application. 3 Steps Choose a default handler or author a custom model handler. You will define a…
Quantization Quantization is a technique to change the data type used to compute neural networks for faster inference. After you’ve deployed your model, there is no need to backpropagate(which is sensitive to precision). This means, if a slight decrease in…
Pruning State-of-the-art deep learning techniques rely on over-parameterized models which makes it hard when the deploying destination has limited resources. Pruning is used to learn the differences between over-parameterized and under-parameterized networks and sparsify your neural networks. In Pytorch, you…
Simple SGD vs Cyclic Learning Rate I compared the training speed between two optimizers by training a UNet Model. Simple SGD optimizer = torch.optim.SGD(model.parameters(), lr=0.01) Cyclic Learning Rate optimizer = torch.optim.SGD(model.parameters(), lr=0.01) scheduler = torch.optim.lr_scheduler.CyclicLR(optimizer, base_lr=0.1, max_lr=1e-4) Cyclic Learning Rate…
Learning Rate Range Test Learning rate may be the most important hyper-parameter in deep learning and you can use this test to find the right learning rate; Run your model and record accuracy/loss for several epochs while letting the learning…