# A Principle of Least Action for the Training of Neural Networks

arXiv: https://arxiv.org/abs/2009.08372 · HTML: https://mlsanity.com/paper/2009.08372/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2020.

## Built on by (most cited first, 8 of 8)

- [Adaptable Hamiltonian neural networks](https://mlsanity.com/paper/2102.13235/) (2021)
- [Mapping conditional distributions for domain adaptation under generalized target shift](https://mlsanity.com/paper/2110.15057/) (2021)
- [Parameter-Efficient Adaptation of Geospatial Foundation Models through Embedding Deflection](https://mlsanity.com/paper/2503.09493/) (2025)
- [Turning Normalizing Flows into Monge Maps with Geodesic Gaussian Preserving Flows](https://mlsanity.com/paper/2209.10873/) (2022)
- [Module-wise Training of Neural Networks via the Minimizing Movement Scheme](https://mlsanity.com/paper/2309.17357/) (2023)
- [Block-wise Training of Residual Networks via the Minimizing Movement Scheme](https://mlsanity.com/paper/2210.00949/) (2022)
- [Adversarial Sample Detection Through Neural Network Transport Dynamics](https://mlsanity.com/paper/2306.04252/) (2023)
- [LaCoOT: Layer Collapse through Optimal Transport](https://mlsanity.com/paper/2406.08933/) (2024)

## Abstract (arXiv)

Neural networks have been achieving high generalization performance on many tasks despite being highly over-parameterized. Since classical statistical learning theory struggles to explain this behavior, much effort has recently been focused on uncovering the mechanisms behind it, in the hope of developing a more adequate theoretical framework and having a better control over the trained models. In this work, we adopt an alternate perspective, viewing the neural network as a dynamical system displacing input particles over time. We conduct a series of experiments and, by analyzing the network's behavior through its displacements, we show the presence of a low kinetic energy displacement bias in the transport map of the network, and link this bias with generalization performance. From this observation, we reformulate the learning problem as follows: finding neural networks which solve the task while transporting the data as efficiently as possible. This offers a novel formulation of the learning problem which allows us to provide regularity results for the solution network, based on Optimal Transport theory. From a practical viewpoint, this allows us to propose a new learning algorithm, which automatically adapts to the complexity of the given task, and leads to networks with a high generalization ability even in low data regimes.