# Finite-time Lyapunov exponents of deep neural networks

arXiv: https://arxiv.org/abs/2306.12548 · HTML: https://mlsanity.com/paper/2306.12548/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 51 references that are in the map.
- First posted on arXiv 21 Jun 2023.

## Builds on (most cited first, 13 of 13)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Exponential expressivity in deep neural networks through transient chaos](https://mlsanity.com/paper/1606.05340/) (2016)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice](https://mlsanity.com/paper/1711.04735/) (2017)
- [Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs](https://mlsanity.com/paper/2112.15383/) (2021)
- [On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems Tools](https://mlsanity.com/paper/2006.14123/) (2020)
- [Geometric compression of invariant manifolds in neural nets](https://mlsanity.com/paper/2007.11471/) (2020)
- [Deep Ensembles from a Bayesian Perspective](https://mlsanity.com/paper/2105.13283/) (2021)
- [Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity](https://mlsanity.com/paper/2010.11775/) (2020)
- [Machine learning with neural networks](https://mlsanity.com/paper/1901.05639/) (2019)

## Built on by (most cited first, 12 of 12)

- [Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topology](https://mlsanity.com/paper/2605.14258/) (2026)
- [Leveraging chaotic transients in the training of artificial neural networks](https://mlsanity.com/paper/2506.08523/) (2025)
- [Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Score](https://mlsanity.com/paper/2606.13300/) (2026)
- [KPFlow: An Operator Perspective on Dynamic Collapse Under Gradient Descent Training of Recurrent Networks](https://mlsanity.com/paper/2507.06381/) (2025)
- [Tracking Finite-Time Lyapunov Exponents to Robustify Neural ODEs](https://mlsanity.com/paper/2602.09613/) (2026)
- [Engineered Ordinary Differential Equations as Classification Algorithm (EODECA): thorough characterization and testing](https://mlsanity.com/paper/2312.14681/) (2023)
- [Attractor-merging Crises and Intermittency in Reservoir Computing](https://mlsanity.com/paper/2504.12695/) (2025)
- [Building causation links in stochastic nonlinear systems from data](https://mlsanity.com/paper/2509.07701/) (2025)
- [Lyapunov Stable Graph Neural Flow](https://mlsanity.com/paper/2603.12557/) (2026)
- [Scalar Representations of Neural Network Training Dynamics](https://mlsanity.com/paper/2606.30384/) (2026)
- [Dynamics to decision: A mathematical theory of Lyapunov spectra and decision boundaries in deep classifiers](https://mlsanity.com/paper/2609.39190/) (2026)
- [Disentangling Computation in Multi-Task Neural Networks with the Green's Operator](https://mlsanity.com/paper/2609.40292/) (2026)

## Abstract (arXiv)

We compute how small input perturbations affect the output of deep neural networks, exploring an analogy between deep networks and dynamical systems, where the growth or decay of local perturbations is characterised by finite-time Lyapunov exponents. We show that the maximal exponent forms geometrical structures in input space, akin to coherent structures in dynamical systems. Ridges of large positive exponents divide input space into different regions that the network associates with different classes. These ridges visualise the geometry that deep networks construct in input space, shedding light on the fundamental mechanisms underlying their learning capabilities.