mlsanityData to 14 Oct 2026

Dynamics to decision: A mathematical theory of Lyapunov spectra and decision boundaries in deep classifiers

arXiv 2609.39190 ↗First posted 30 Sep 2026cs.LG cs.SY eess.SYcited by 0

Not on the map. Why?

Abstract (arXiv)

A deep classifier is defined not only by the decision it produces, but also by the sequence of transformations through which that decision is formed. Treating this evolution as a dynamical system across layers provides a natural framework for asking how decision geometry emerges through depth and how far back we can trace a boundary's dynamical signature. We model a feed-forward classifier as a finite, nonautonomous discrete dynamical system, with layers playing the role of discrete time steps. We study the Finite-Time Maximum Lyapunov Exponent (FTMLE) of the data samples' dynamical trajectory through depths of the classifier. The FTMLE measures the rate of convergence/divergence of nearby trajectories. We move the observation endpoint backward from probabilities to logits and then to hidden representations. For Gaussian classes, we prove that probability-level FTMLE carries a clear geometric signature of the decision boundary, with its dominant direction aligned with the boundary normal. Moving one step backward to the logits, we prove this relationship is no longer universal but depends critically on how the classifier is trained, particularly on the choice of loss function. Moving further backward to the hidden representation, the connection becomes more conditional: boundary-related FTMLE can persist, but only under identifiable structural conditions. We propose geometry-aware fine-tuning for restructuring the classifier's hidden FTMLE, and propose conditions for guaranteed concentration of high hidden FTMLE near the decision boundary. Through our numerical results, we show the generality and validity of our theoretical results. Understanding the evolution of data samples as traveling through the layers of classifier provides a principled foundation for identifying where boundary-relevant sensitivity emerges and for developing layer-aware regularization strategies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 57 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

45 of 57 references.

References · 45

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  3. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  4. 2015The Limitations of Deep Learning in Adversarial Settingscited by 790
  5. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  6. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  7. 2017Stable Architectures for Deep Neural Networkscited by 341
  8. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
  9. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
  10. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
  11. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  12. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
33 more
  1. 2018Sorting out Lipschitz function approximationcited by 213
  2. 2016Deep Information Propagationcited by 195
  3. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 195
  4. 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 189
  5. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 185
  6. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 182
  7. 2018Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approachcited by 182
  8. 2017Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practicecited by 132
  9. 2019Robust Learning with Jacobian Regularizationcited by 129
  10. 2016Robust Large Margin Deep Neural Networkscited by 118
  11. 2016Robustness of classifiers: from adversarial to random noisecited by 88
  12. 2018Improving DNN Robustness to Adversarial Attacks using Jacobian Regularizationcited by 88
  13. 2018The Emergence of Spectral Universality in Deep Networkscited by 74
  14. 2021A Dynamical System Perspective for Lipschitz Neural Networkscited by 60
  15. 2017Art of singular vectors and universal adversarial perturbationscited by 49
  16. 2019Scaleable input gradient regularization for adversarial robustnesscited by 47
  17. 2023SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundariescited by 31
  18. 2021Adversarial Robustness via Fisher-Rao Regularizationcited by 19
  19. 2018The Adversarial Attack and Detection under the Fisher Information Metriccited by 18
  20. 2020On 1/n neural representation and robustnesscited by 17
  21. 2024Gradient-Regularized Out-of-Distribution Detectioncited by 13
  22. 2023Finite-time Lyapunov exponents of deep neural networkscited by 12
  23. 2019Defending Against Adversarial Attacks by Suppressing the Largest Eigenvalue of Fisher Information Matrixcited by 9
  24. 2022Adversarial attacks on neural networks through canonical Riemannian foliationscited by 5
  25. 2022Exact Spectral Norm Regularization for Neural Networkscited by 5
  26. 2023How does training shape the Riemannian geometry of neural network representations?cited by 5
  27. 2023Characterizing Data Point Vulnerability via Average-Case Robustnesscited by 3
  28. 2022Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separationcited by 2
  29. 2024Spectral regularization for adversarially-robust representation learningcited by 2
  30. 2024Adversarially robust generalization theory via Jacobian regularization for deep neural networkscited by 1
  31. 2025Category learning in deep neural networks: Information content and geometry of internal representationscited by 1
  32. 2026Tracking Finite-Time Lyapunov Exponents to Robustify Neural ODEscited by 1
  33. 2026Measuring Model Robustness via Fisher Information: Spectral Bounds, Theoretical Guarantees, and Practical Algorithmscited by 1

Built on by

Not cited yet.