Integration Methods and Accelerated Optimization Algorithms
Not on the map. Why?
Abstract (arXiv)
We show that accelerated optimization methods can be seen as particular instances of multi-step integration schemes from numerical analysis, applied to the gradient flow equation. In comparison with recent advances in this vein, the differential equation considered here is the basic gradient flow and we show that multi-step schemes allow integration of this differential equation using larger step sizes, thus intuitively explaining acceleration results.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Feb 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 2; 2022: 2; 2023: 2.
6 papers.
- 2022Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equationscited by 42
- 2021Continuous vs. Discrete Optimization of Deep Neural Networkscited by 34
- 2023Geometry-aware training of factorized layers in tensor Tucker formatcited by 12
- 2021Potential Function-based Framework for Making the Gradients Small in Convex and Min-Max Optimizationcited by 4
- 2022Ensemble Reinforcement Learning in Continuous Spaces -- A Hierarchical Multi-Step Approach for Policy Trainingcited by 1
- 2023On Tuning Neural ODE for Stability, Consistency and Faster Convergencecited by 1
- 2023On Tuning Neural ODE for Stability, Consistency and Faster Convergence4 Dec 2023
- 2023Geometry-aware training of factorized layers in tensor Tucker format30 May 2023
- 2022Ensemble Reinforcement Learning in Continuous Spaces -- A Hierarchical Multi-Step Approach for Policy Training29 Sep 2022
- 2022Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations26 May 2022
- 2021Continuous vs. Discrete Optimization of Deep Neural Networks14 Jul 2021
- 2021Potential Function-based Framework for Making the Gradients Small in Convex and Min-Max Optimization28 Jan 2021