# Born-Infeld (BI) for AI: Energy-Conserving Descent (ECD) for Optimization

arXiv: https://arxiv.org/abs/2201.11137 · HTML: https://mlsanity.com/paper/2201.11137/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 26 Jan 2022.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Measuring the Intrinsic Dimension of Objective Landscapes](https://mlsanity.com/paper/1804.08838/) (2018)
- [Essentially No Barriers in Neural Network Energy Landscape](https://mlsanity.com/paper/1803.00885/) (2018)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [The large learning rate phase of deep learning: the catapult mechanism](https://mlsanity.com/paper/2003.02218/) (2020)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [The Principles of Deep Learning Theory](https://mlsanity.com/paper/2106.10165/) (2021)
- [Why Deep Neural Networks for Function Approximation?](https://mlsanity.com/paper/1610.04161/) (2016)
- [A Literature Survey of Benchmark Functions For Global Optimization Problems](https://mlsanity.com/paper/1308.4008/) (2013)
- [Manipulating SGD with Data Ordering Attacks](https://mlsanity.com/paper/2104.09667/) (2021)
- [The gap between theory and practice in function approximation with deep neural networks](https://mlsanity.com/paper/2001.07523/) (2020)
- [Fluctuation-dissipation relations for stochastic gradient descent](https://mlsanity.com/paper/1810.00004/) (2018)
- [Solving the Kolmogorov PDE by means of deep learning](https://mlsanity.com/paper/1806.00421/) (2018)
- [Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimality](https://mlsanity.com/paper/2110.06897/) (2021)
- [The Goldilocks zone: Towards better understanding of neural network loss landscapes](https://mlsanity.com/paper/1807.02581/) (2018)
- [Conformal Symplectic and Relativistic Optimization](https://mlsanity.com/paper/1903.04100/) (2019)
- [Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives](https://mlsanity.com/paper/2002.12493/) (2020)
- [Numerical Solution of the Parametric Diffusion Equation by Deep Neural Networks](https://mlsanity.com/paper/2004.12131/) (2020)
- [The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion](https://mlsanity.com/paper/2107.09133/) (2021)
- [Relativistic Monte Carlo](https://mlsanity.com/paper/1609.04388/) (2016)

## Built on by (most cited first, 10 of 10)

- [A duality connecting neural network and cosmological dynamics](https://mlsanity.com/paper/2202.11104/) (2022)
- [TASI Lectures on Physics for Machine Learning](https://mlsanity.com/paper/2408.00082/) (2024)
- [Symmetry Breaking in Transformers for Efficient and Interpretable Training](https://mlsanity.com/paper/2601.22257/) (2026)
- [Improving Energy Conserving Descent for Machine Learning: Theory and Practice](https://mlsanity.com/paper/2306.00352/) (2023)
- [Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimization](https://mlsanity.com/paper/2307.10053/) (2023)
- [Statistical Properties of Training & Generalization](https://mlsanity.com/paper/2606.20299/) (2026)
- [Probing optimisation in physics-informed neural networks](https://mlsanity.com/paper/2303.15196/) (2023)
- [The Optimiser Hidden in Plain Sight: Training with the Loss Landscape's Induced Metric](https://mlsanity.com/paper/2509.03594/) (2025)
- [Classical and Quantum Speedups for Non-Convex Optimization via Energy Conserving Descent](https://mlsanity.com/paper/2604.13022/) (2026)
- [A Physical Response-and-Memory Model for Muon Optimization](https://mlsanity.com/paper/2608.22994/) (2026)

## Abstract (arXiv)

We introduce a novel framework for optimization based on energy-conserving Hamiltonian dynamics in a strongly mixing (chaotic) regime and establish its key properties analytically and numerically. The prototype is a discretization of Born-Infeld dynamics, with a squared relativistic speed limit depending on the objective function. This class of frictionless, energy-conserving optimizers proceeds unobstructed until slowing naturally near the minimal loss, which dominates the phase space volume of the system. Building from studies of chaotic systems such as dynamical billiards, we formulate a specific algorithm with good performance on machine learning and PDE-solving tasks, including generalization. It cannot stop at a high local minimum, an advantage in non-convex loss functions, and proceeds faster than GD+momentum in shallow valleys.