mlsanityData to 14 Oct 2026

Finite-time Lyapunov exponents of deep neural networks

arXiv 2306.12548 ↗First posted 21 Jun 2023cond-mat.dis-nn cs.LG stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

We compute how small input perturbations affect the output of deep neural networks, exploring an analogy between deep networks and dynamical systems, where the growth or decay of local perturbations is characterised by finite-time Lyapunov exponents. We show that the maximal exponent forms geometrical structures in input space, akin to coherent structures in dynamical systems. Ridges of large positive exponents divide input space into different regions that the network associates with different classes. These ridges visualise the geometry that deep networks construct in input space, shedding light on the fundamental mechanisms underlying their learning capabilities.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 13 of its 51 references that are in the map. First posted on arXiv 21 Jun 2023.

Builds on

13 of 51 references.

References · 13

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  4. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  5. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
  6. 2019Universal Approximation with Deep Narrow Networkscited by 200
  7. 2017Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practicecited by 132
  8. 2021Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNscited by 53
  9. 2020On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems Toolscited by 31
  10. 2020Geometric compression of invariant manifolds in neural netscited by 28
  11. 2021Deep Ensembles from a Bayesian Perspectivecited by 27
  12. 2020Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticitycited by 18
1 more
  1. 2019Machine learning with neural networkscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 4; 2026: 7.

12 papers.

  1. 2026Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topologycited by 5
  2. 2025Leveraging chaotic transients in the training of artificial neural networkscited by 3
  3. 2026Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Scorecited by 2
  4. 2025KPFlow: An Operator Perspective on Dynamic Collapse Under Gradient Descent Training of Recurrent Networkscited by 1
  5. 2026Tracking Finite-Time Lyapunov Exponents to Robustify Neural ODEscited by 1
  6. 2023Engineered Ordinary Differential Equations as Classification Algorithm (EODECA): thorough characterization and testingcited by 0
  7. 2025Attractor-merging Crises and Intermittency in Reservoir Computingcited by 0
  8. 2025Building causation links in stochastic nonlinear systems from datacited by 0
  9. 2026Lyapunov Stable Graph Neural Flowcited by 0
  10. 2026Scalar Representations of Neural Network Training Dynamicscited by 0
  11. 2026Dynamics to decision: A mathematical theory of Lyapunov spectra and decision boundaries in deep classifierscited by 0
  12. 2026Disentangling Computation in Multi-Task Neural Networks with the Green's Operatorcited by 0