mlsanityData to 7 Oct 2026

From Adam to Adam-Like Lagrangians: Second-Order Nonlocal Dynamics

arXiv 2602.09101 ↗First posted 9 Feb 2026cs.LG cs.NA math.DScited by 1

Not on the map. Why?

Abstract (arXiv)

In this paper, we derive an accelerated continuous-time formulation of Adam by modeling it as a second-order integro-differential dynamical system. We relate this inertial nonlocal model to an existing first-order nonlocal Adam flow through an α-refinement limit, and we provide Lyapunov-based stability and convergence analyses. We also introduce an Adam-inspired nonlocal Lagrangian formulation, offering a variational viewpoint. Numerical simulations on Rosenbrock-type examples show agreement between the proposed dynamics and discrete Adam.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 38 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

16 of 38 references.

References · 16

  1. 2021Convex optimizationcited by 2,479
  2. 2019On the Convergence of Adam and Beyondcited by 834
  3. 2019Hamiltonian Neural Networkscited by 702
  4. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  5. 2020Lagrangian Neural Networkscited by 388
  6. 2015A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insightscited by 203
  7. 2016A Variational Perspective on Accelerated Methods in Optimizationcited by 108
  8. 2018Convergence and Dynamical Behavior of the ADAM Algorithm for Non-Convex Stochastic Optimizationcited by 47
  9. 2021Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networkscited by 47
  10. 2018A general system of differential equations to model first order adaptive algorithmscited by 28
  11. 2021Noether Networks: Meta-Learning Useful Conserved Quantitiescited by 26
  12. 2016Stochastic Gradient Descent in Continuous Timecited by 14
4 more
  1. 2024Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equationscited by 6
  2. 2020Optimization and Learning With Nonlocal Calculuscited by 5
  3. 2024Convergence of continuous-time stochastic gradient descent with applications to deep neural networkscited by 4
  4. 2024Continuous-Time Analysis of Adaptive Optimization and Normalizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Deterministic Adam-Inspired Methods with Accelerated Convergence Ratecited by 1