# Selection dynamics for deep neural networks

arXiv: https://arxiv.org/abs/1905.09076 · HTML: https://mlsanity.com/paper/1905.09076/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2019.

## Built on by (most cited first, 6 of 6)

- [Turnpike in optimal control of PDEs, ResNets, and beyond](https://mlsanity.com/paper/2202.04097/) (2022)
- [Online Control-Informed Learning](https://mlsanity.com/paper/2410.03924/) (2024)
- [An Optimal Time Variable Learning Framework for Deep Neural Networks](https://mlsanity.com/paper/2204.08528/) (2022)
- [From NeurODEs to AutoencODEs: a mean-field control framework for width-varying Neural Networks](https://mlsanity.com/paper/2307.02279/) (2023)
- [Understanding the role of autoencoders for stiff dynamical systems using information theory](https://mlsanity.com/paper/2503.06325/) (2025)
- [Time Regularization in Optimal Time Variable Learning](https://mlsanity.com/paper/2306.16111/) (2023)

## Abstract (arXiv)

This paper presents a partial differential equation framework for deep residual neural networks and for the associated learning problem. This is done by carrying out the continuum limits of neural networks with respect to width and depth. We study the wellposedness, the large time solution behavior, and the characterization of the steady states of the forward problem. Several useful time-uniform estimates and stability/instability conditions are presented. We state and prove optimality conditions for the inverse deep learning problem, using standard variational calculus, the Hamilton-Jacobi-Bellmann equation and the Pontryagin maximum principle. This serves to establish a mathematical foundation for investigating the algorithmic and theoretical connections between neural networks, PDE theory, variational analysis, optimal control, and deep learning.