eagle: early approximated gradient based learning rate estimator
Not on the map. Why?
Abstract (arXiv)
We propose EAGLE update rule, a novel optimization method that accelerates loss convergence during the early stages of training by leveraging both current and previous step parameter and gradient values. The update algorithm estimates optimal parameters by computing the changes in parameters and gradients between consecutive training steps and leveraging the local curvature of the loss landscape derived from these changes. However, this update rule has potential instability, and to address that, we introduce an adaptive switching mechanism that dynamically selects between Adam and EAGLE update rules to enhance training stability. Experiments on standard benchmark datasets demonstrate that EAGLE optimizer, which combines this novel update rule with the switching mechanism achieves rapid training loss convergence with fewer epochs, compared to conventional optimization methods.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). First posted on arXiv 3 Feb 2025.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2026: 1.
1 paper.