Polynomial Learning Rate
For deep learning models, the learning rate is one of the most important hyper-parameters in any deep neural network optimization process.
Polynomial Learning Rate is a proposed technique to apply learning rate decay and optimize such process.
Hyper-parameters
There are 4 hyper-parameters.
1. Lro: The initial learning rate
2. I: Current number of iteration
3. Ti: Total number of iteration
4. Power: Controls the shape of the learning rate decay
Results
The research used 0.9 for the power parameter and was able to achieve faster learning and higher accuracy.
