mlsanityData to 7 Oct 2026

Born-Infeld (BI) for AI: Energy-Conserving Descent (ECD) for Optimization

arXiv 2201.11137 ↗First posted 26 Jan 2022cs.LG astro-ph.CO hep-thcited by 10

Not on the map. Why?

Abstract (arXiv)

We introduce a novel framework for optimization based on energy-conserving Hamiltonian dynamics in a strongly mixing (chaotic) regime and establish its key properties analytically and numerically. The prototype is a discretization of Born-Infeld dynamics, with a squared relativistic speed limit depending on the objective function. This class of frictionless, energy-conserving optimizers proceeds unobstructed until slowing naturally near the minimal loss, which dominates the phase space volume of the system. Building from studies of chaotic systems such as dynamical billiards, we formulate a specific algorithm with good performance on machine learning and PDE-solving tasks, including generalization. It cannot stop at a high local minimum, an advantage in non-convex loss functions, and proceeds faster than GD+momentum in shallow valleys.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 27 of its 56 references that are in the map. First posted on arXiv 26 Jan 2022.

Builds on

27 of 56 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  4. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  5. 2019On the Convergence of Adam and Beyondcited by 834
  6. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  7. 2019DeepXDE: A deep learning library for solving differential equationscited by 617
  8. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  9. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
  10. 2018Essentially No Barriers in Neural Network Energy Landscapecited by 328
  11. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  12. 2017A Conceptual Introduction to Hamiltonian Monte Carlocited by 207
15 more
  1. 2020The large learning rate phase of deep learning: the catapult mechanismcited by 206
  2. 2021The Principles of Deep Learning Theorycited by 169
  3. 2016Why Deep Neural Networks for Function Approximation?cited by 82
  4. 2013A Literature Survey of Benchmark Functions For Global Optimization Problemscited by 75
  5. 2021Manipulating SGD with Data Ordering Attackscited by 73
  6. 2020The gap between theory and practice in function approximation with deep neural networkscited by 53
  7. 2018Fluctuation-dissipation relations for stochastic gradient descentcited by 47
  8. 2018Solving the Kolmogorov PDE by means of deep learningcited by 41
  9. 2021Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimalitycited by 40
  10. 2018The Goldilocks zone: Towards better understanding of neural network loss landscapescited by 32
  11. 2019Conformal Symplectic and Relativistic Optimizationcited by 28
  12. 2020Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectivescited by 27
  13. 2020Numerical Solution of the Parametric Diffusion Equation by Deep Neural Networkscited by 27
  14. 2021The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusioncited by 19
  15. 2016Relativistic Monte Carlocited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 1; 2025: 1; 2026: 4.

10 papers.

  1. 2022A duality connecting neural network and cosmological dynamicscited by 10
  2. 2024TASI Lectures on Physics for Machine Learningcited by 4
  3. 2023Improving Energy Conserving Descent for Machine Learning: Theory and Practicecited by 3
  4. 2023Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimizationcited by 3
  5. 2026Symmetry Breaking in Transformers for Efficient and Interpretable Trainingcited by 2
  6. 2026Statistical Properties of Training & Generalizationcited by 2
  7. 2023Probing optimisation in physics-informed neural networkscited by 1
  8. 2025The Optimiser Hidden in Plain Sight: Training with the Loss Landscape's Induced Metriccited by 1
  9. 2026Classical and Quantum Speedups for Non-Convex Optimization via Energy Conserving Descentcited by 0
  10. 2026A Physical Response-and-Memory Model for Muon Optimizationcited by 0