# From Adam to Adam-Like Lagrangians: Second-Order Nonlocal Dynamics

arXiv: https://arxiv.org/abs/2602.09101 · HTML: https://mlsanity.com/paper/2602.09101/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 16 of 16)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/1608.04636/) (2016)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015)
- [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016)
- [Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks](https://mlsanity.com/paper/2105.02716/) (2021)
- [Convergence and Dynamical Behavior of the ADAM Algorithm for Non-Convex Stochastic Optimization](https://mlsanity.com/paper/1810.02263/) (2018)
- [A general system of differential equations to model first order adaptive algorithms](https://mlsanity.com/paper/1810.13108/) (2018)
- [Noether Networks: Meta-Learning Useful Conserved Quantities](https://mlsanity.com/paper/2112.03321/) (2021)
- [Stochastic Gradient Descent in Continuous Time](https://mlsanity.com/paper/1611.05545/) (2016)
- [Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations](https://mlsanity.com/paper/2411.09734/) (2024)
- [Optimization and Learning With Nonlocal Calculus](https://mlsanity.com/paper/2012.07013/) (2020)
- [Convergence of continuous-time stochastic gradient descent with applications to deep neural networks](https://mlsanity.com/paper/2409.07401/) (2024)
- [Continuous-Time Analysis of Adaptive Optimization and Normalization](https://mlsanity.com/paper/2411.05746/) (2024)

## Built on by (most cited first, 1 of 1)

- [Deterministic Adam-Inspired Methods with Accelerated Convergence Rate](https://mlsanity.com/paper/2604.08742/) (2026)

## Abstract (arXiv)

In this paper, we derive an accelerated continuous-time formulation of Adam by modeling it as a second-order integro-differential dynamical system. We relate this inertial nonlocal model to an existing first-order nonlocal Adam flow through an α-refinement limit, and we provide Lyapunov-based stability and convergence analyses. We also introduce an Adam-inspired nonlocal Lagrangian formulation, offering a variational viewpoint. Numerical simulations on Rosenbrock-type examples show agreement between the proposed dynamics and discrete Adam.