mlsanityData to 7 Oct 2026

A Dynamical System Perspective for Lipschitz Neural Networks

arXiv 2110.12690 ↗First posted 25 Oct 2021cs.LGcited by 59

Not on the map. Why?

Abstract (arXiv)

The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building 1-Lipschitz Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method to build 1-Lipschitz Neural Networks and show that some previous approaches are special cases of this framework. Then, we extend this reasoning and show that ResNet flows derived from convex potentials define 1-Lipschitz transformations, that lead us to define the em Convex Potential Layer (CPL). A comprehensive set of experiments on several datasets demonstrates the scalability of our architecture and the benefits as an ℓ2-provable defense against adversarial examples.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Builds on 57 of its 76 references that are in the map. First posted on arXiv 25 Oct 2021.

Builds on

57 of 76 references.

References · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2013Intriguing properties of neural networkscited by 5,114
  9. 2018Neural Ordinary Differential Equationscited by 3,733
  10. 2015Variational Inference with Normalizing Flowscited by 2,161
  11. 2016Adversarial examples in the physical worldcited by 1,762
  12. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
38 more
  1. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  2. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  3. 2019Hamiltonian Neural Networkscited by 702
  4. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  5. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  6. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  7. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  8. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 415
  9. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  10. 2016Input Convex Neural Networkscited by 407
  11. 2018Invertible Residual Networkscited by 364
  12. 2017Stable Architectures for Deep Neural Networkscited by 340
  13. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
  14. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
  15. 2018Certified Defenses against Adversarial Examplescited by 305
  16. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  17. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
  18. 2018Sorting out Lipschitz function approximationcited by 211
  19. 2017Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equationscited by 203
  20. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 193
  21. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 178
  22. 2018Certified Adversarial Robustness with Additive Noisecited by 178
  23. 2018Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networkscited by 176
  24. 2018Robustness via curvature regularization, and vice versacited by 174
  25. 2017Towards Robust Neural Networks via Random Self-ensemblecited by 158
  26. 2020Randomized Smoothing of All Shapes and Sizescited by 146
  27. 2018Scaling provable adversarial defensescited by 141
  28. 2019Orthogonal Convolutional Neural Networkscited by 122
  29. 2021Orthogonalizing Convolutional Layers with the Cayley Transformcited by 105
  30. 2018The Singular Values of Convolutional Layerscited by 95
  31. 2020Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimizationcited by 89
  32. 2019Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networkscited by 86
  33. 2018Sylvester Normalizing Flows for Variational Inferencecited by 71
  34. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  35. 2021Skew Orthogonal Convolutionscited by 67
  36. 2017Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classificationcited by 56
  37. 2019Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operatorscited by 54
  38. 2020Stable ResNetcited by 49

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 19; 2024: 9; 2025: 14; 2026: 13.

59 papers.

Sort:

By area:

  1. 2023A Unified Algebraic Perspective on Lipschitz Neural Networkscited by 58
  2. 2023Direct Parameterization of Lipschitz-Bounded Deep Networkscited by 49
  3. 2022Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspectivecited by 48
  4. 2023R-LPIPS: An Adversarially Robust Perceptual Similarity Metriccited by 27
  5. 2023Some Fundamental Aspects about Lipschitz Continuity of Neural Networkscited by 25
  6. 2023LipSim: A Provably Robust Perceptual Similarity Metriccited by 15
  7. 20231-Lipschitz Layers Compared: Memory, Speed, and Certifiable Robustnesscited by 15
  8. 2023Unlocking Deterministic Robustness Certification on ImageNetcited by 13
  9. 2023Robust low-rank training via approximate orthonormal constraintscited by 12
  10. 2023DP-SGD Without Clipping: The Lipschitz Neural Network Waycited by 11
  11. 2025SDP-CROWN: Efficient Bound Propagation for Neural Network Verification with Tightness of Semidefinite Programmingcited by 10
  12. 2023Understanding Self-attention Mechanism via Dynamical System Perspectivecited by 9
38 more
  1. 2024Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networkscited by 9
  2. 20241-Lipschitz Neural Distance Fieldscited by 8
  3. 2025An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architecturescited by 8
  4. 2022On the Safety of Interpretable Machine Learning: A Maximum Deviation Approachcited by 7
  5. 2024Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layerscited by 7
  6. 2025Approximation theory for 1-Lipschitz ResNetscited by 7
  7. 2023The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothingcited by 6
  8. 2024Novel Quadratic Constraints for Extending LipSDP beyond Slope-Restricted Activationscited by 6
  9. 2022A Generic Shared Attention Mechanism for Various Backbone Neural Networkscited by 5
  10. 2025Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Losscited by 5
  11. 2026Approximation Theory for Lipschitz Continuous Transformerscited by 5
  12. 2023AttNS: Attention-Inspired Numerical Solving For Limited Data Scenarioscited by 4
  13. 2023Differentially Private Low-dimensional Synthetic Data from High-dimensional Datasetscited by 4
  14. 2023A Recipe for Improved Certifiable Robustnesscited by 4
  15. 2023Certification of Deep Learning Models for Medical Image Segmentationcited by 4
  16. 2024Spectral Norm of Convolutional Layers with Circular and Zero Paddingscited by 4
  17. 2024Online Differentially Private Synthetic Data Generationcited by 4
  18. 2025Efficient Robust Conformal Prediction via Lipschitz-Bounded Networkscited by 4
  19. 2023Certified Robust Models with Slack Control and Large Lipschitz Constantscited by 3
  20. 2024Intriguing Properties of Robust Classificationcited by 3
  21. 2025LipShiFT: A Certifiably Robust Shift-based Vision Transformercited by 3
  22. 2022Towards Evading the Limits of Randomized Smoothing: A Theoretical Analysiscited by 2
  23. 2023Predictions Based on Pixel Data: Insights from PDEs and Finite Differencescited by 2
  24. 2023Towards Better Certified Segmentation via Diffusion Modelscited by 2
  25. 2024Learning conditional distributions on continuous spacescited by 2
  26. 20251-Lipschitz Network Initialization for Certifiably Robust Classification Applications: A Decay Problemcited by 2
  27. 2025Bridging the Theoretical Gap in Randomized Smoothingcited by 2
  28. 2026LipNeXt: Scaling up Lipschitz-based Certified Robustness to Billion-parameter Modelscited by 2
  29. 20231-Lipschitz Neural Networks are more expressive with N-Activationscited by 1
  30. 2024Integrating uncertainty quantification into randomized smoothing based robustness guaranteescited by 1
  31. 2026Learning Better Certified Models from Empirically-Robust Teacherscited by 1
  32. 2026LipsAM: Lipschitz-Continuous Amplitude Modifier for Audio Signal Processing and its Application to Plug-and-Play Dereverberationcited by 1
  33. 2025L-Lipschitz Gershgorin ResNet Networkcited by 0
  34. 2025APECS: Adaptive Personalized Control System Architecturecited by 0
  35. 2025Deep Network Trainability via Persistent Subspace Orthogonalitycited by 0
  36. 2025Stable neural networks and connections to continuous dynamical systemscited by 0
  37. 2025Delving into Cascaded Instability: A Lipschitz Continuity View on Image Restoration and Object Detection Synergycited by 0
  38. 2025LDLT L-Lipschitz Network: Generalized Deep End-To-End Lipschitz Network Constructioncited by 0

50 most cited of 59 All 59 (JSON)