# Improved Binary Forward Exploration: Learning Rate Scheduling Method for Stochastic Optimization

arXiv: https://arxiv.org/abs/2207.04198 · HTML: https://mlsanity.com/paper/2207.04198/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 33 references that are in the map.
- First posted on arXiv 9 Jul 2022.

## Builds on (most cited first, 6 of 6)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Aggregated Momentum: Stability Through Passive Damping](https://mlsanity.com/paper/1804.00325/) (2018)
- [Convergence Analysis of Gradient Descent Algorithms with Proportional Updates](https://mlsanity.com/paper/1801.03137/) (2018)
- [Machine Learning Solar Wind Driving Magnetospheric Convection in Tail Lobes](https://mlsanity.com/paper/2202.01383/) (2022)
- [BFE and AdaBFE: A New Approach in Learning Rate Automation for Stochastic Optimization](https://mlsanity.com/paper/2207.02763/) (2022)

## Abstract (arXiv)

A new gradient-based optimization approach by automatically scheduling the learning rate has been proposed recently, which is called Binary Forward Exploration (BFE). The Adaptive version of BFE has also been discussed thereafter. In this paper, the improved algorithms based on them will be investigated, in order to optimize the efficiency and robustness of the new methodology. This improved approach provides a new perspective to scheduling the update of learning rate and will be compared with the stochastic gradient descent, aka SGD algorithm with momentum or Nesterov momentum and the most successful adaptive learning rate algorithm e.g. Adam. The goal of this method does not aim to beat others but provide a different viewpoint to optimize the gradient descent process. This approach combines the advantages of the first-order and second-order optimizations in the aspects of speed and efficiency.