mlsanityData to 7 Oct 2026

A Unified Algebraic Perspective on Lipschitz Neural Networks

arXiv 2303.03169 ↗First posted 6 Mar 2023cs.LGcited by 58

Not on the map. Why?

Abstract (arXiv)

Important research efforts have focused on the design and training of neural networks with a controlled Lipschitz constant. The goal is to increase and sometimes guarantee the robustness against adversarial attacks. Recent promising techniques draw inspirations from different backgrounds to design 1-Lipschitz neural networks, just to name a few: convex potential layers derive from the discretization of continuous dynamical systems, Almost-Orthogonal-Layer proposes a tailored method for matrix rescaling. However, it is today important to consider the recent and promising contributions in the field under a common theoretical lens to better design new and improved layers. This paper introduces a novel algebraic perspective unifying various types of 1-Lipschitz neural networks, including the ones previously mentioned, along with methods based on orthogonality and spectral methods. Interestingly, we show that many existing techniques can be derived and generalized via finding analytical solutions of a common semidefinite programming (SDP) condition. We also prove that AOL biases the scaled weight to the ones which are close to the set of orthogonal matrices in a certain mathematical manner. Moreover, our algebraic condition, combined with the Gershgorin circle theorem, readily leads to new and diverse parameterizations for 1-Lipschitz network layers. Our approach, called SDP-based Lipschitz Layers (SLL), allows us to design non-trivial yet efficient generalization of convex potential layers. Finally, the comprehensive set of experiments on image classification shows that SLLs outperform previous approaches on certified robust accuracy. Code is available at https://github.com/araujoalexandre/Lipschitz-SLL-Networks.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Builds on 30 of its 38 references that are in the map. First posted on arXiv 6 Mar 2023.

Builds on

30 of 38 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  3. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  4. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  5. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  6. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  7. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  8. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
  9. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  10. 2018Sorting out Lipschitz function approximationcited by 211
  11. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 193
  12. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
18 more
  1. 2018Semidefinite relaxations for certifying robustness to adversarial examplescited by 178
  2. 2020Randomized Smoothing of All Shapes and Sizescited by 146
  3. 2021Globally-Robust Neural Networkscited by 106
  4. 2021Orthogonalizing Convolutional Layers with the Cayley Transformcited by 105
  5. 2019Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programmingcited by 100
  6. 2020Training robust neural networks using Lipschitz boundscited by 98
  7. 2019Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networkscited by 86
  8. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  9. 2021Skew Orthogonal Convolutionscited by 67
  10. 2021Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Boundscited by 64
  11. 2021A Dynamical System Perspective for Lipschitz Neural Networkscited by 59
  12. 2020Curse of Dimensionality on Randomized Smoothing for Certifiable Robustnesscited by 56
  13. 2020Lipschitz Bounded Equilibrium Networkscited by 49
  14. 2021Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100cited by 47
  15. 2022Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networkscited by 39
  16. 2022LOT: Layer-wise Orthogonal Training on Improving ℓ2 Certified Robustnesscited by 32
  17. 2022A Quantitative Geometric Approach to Neural-Network Smoothnesscited by 16
  18. 2022Improved techniques for deterministic l2 robustnesscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 17; 2025: 17; 2026: 9.

58 papers.

Sort:

By area:

  1. 2023R-LPIPS: An Adversarially Robust Perceptual Similarity Metriccited by 27
  2. 2023Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularizationcited by 19
  3. 2023On the Robustness of Removal-Based Feature Attributionscited by 18
  4. 2023LipSim: A Provably Robust Perceptual Similarity Metriccited by 15
  5. 20231-Lipschitz Layers Compared: Memory, Speed, and Certifiable Robustnesscited by 15
  6. 2024ECLipsE: Efficient Compositional Lipschitz Constant Estimation for Deep Neural Networkscited by 10
  7. 2024Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifierscited by 10
  8. 2025Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metricscited by 10
  9. 2025SDP-CROWN: Efficient Bound Propagation for Neural Network Verification with Tightness of Semidefinite Programmingcited by 10
  10. 2023Lipschitz-bounded 1D convolutional neural networks using the Cayley transform and the controllability Gramiancited by 9
  11. 2024Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networkscited by 9
  12. 2024Lipschitz constant estimation for general neural network architectures using control toolscited by 8
38 more
  1. 20241-Lipschitz Neural Distance Fieldscited by 8
  2. 2025An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architecturescited by 8
  3. 2023Learning Decentralized Partially Observable Mean Field Control for Artificial Collective Behaviorcited by 6
  4. 2023The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothingcited by 6
  5. 2024Novel Quadratic Constraints for Extending LipSDP beyond Slope-Restricted Activationscited by 6
  6. 2025Neural Contraction Metrics with Formal Guarantees for Discrete-Time Nonlinear Dynamical Systemscited by 5
  7. 2025Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Losscited by 5
  8. 2023Major-Minor Mean Field Multi-Agent Reinforcement Learningcited by 4
  9. 2023A Recipe for Improved Certifiable Robustnesscited by 4
  10. 2023Certification of Deep Learning Models for Medical Image Segmentationcited by 4
  11. 2024Spectral Norm of Convolutional Layers with Circular and Zero Paddingscited by 4
  12. 2024On Robust Reinforcement Learning with Lipschitz-Bounded Policy Networkscited by 4
  13. 2024The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspectivecited by 4
  14. 2025Efficient Robust Conformal Prediction via Lipschitz-Bounded Networkscited by 4
  15. 2023Certified Robust Models with Slack Control and Large Lipschitz Constantscited by 3
  16. 2024Compositional Curvature Bounds for Deep Neural Networkscited by 3
  17. 2025LipShiFT: A Certifiably Robust Shift-based Vision Transformercited by 3
  18. 2023Towards Better Certified Segmentation via Diffusion Modelscited by 2
  19. 2024Learning conditional distributions on continuous spacescited by 2
  20. 2024A Complete Set of Quadratic Constraints for Repeated ReLU and Generalizationscited by 2
  21. 2024LipKernel: Lipschitz-Bounded Convolutional Neural Networks via Dissipative Layerscited by 2
  22. 2024Certified Robustness for Deep Equilibrium Models via Serialized Random Smoothingcited by 2
  23. 20251-Lipschitz Network Initialization for Certifiably Robust Classification Applications: A Decay Problemcited by 2
  24. 2025Improved Scalable Lipschitz Bounds for Deep Neural Networkscited by 2
  25. 2025R2DN: Scalable Parameterization of Contracting and Lipschitz Recurrent Deep Networkscited by 2
  26. 2025Bridging the Theoretical Gap in Randomized Smoothingcited by 2
  27. 2025Floating-Point Neural Network Verification at the Software Levelcited by 2
  28. 2026LipNeXt: Scaling up Lipschitz-based Certified Robustness to Billion-parameter Modelscited by 2
  29. 2023Efficient Symbolic Reasoning for Neural-Network Verificationcited by 1
  30. 2023Is Certifying ℓp Robustness Still Worthwhile?cited by 1
  31. 2024A Unified Framework for Neural Computation and Learning Over Timecited by 1
  32. 2024Learning to Stop: Deep Learning for Mean Field Optimal Stoppingcited by 1
  33. 2025ECLipsE-Gen-Local: Efficient Compositional Local Lipschitz Estimates for Deep Neural Networkscited by 1
  34. 2026Controller Design for Structured State-space Models via Contraction Theorycited by 1
  35. 2026Certified vs. Empirical Adversarial Robust-ness via Hybrid Convolutions with Attention Stochasticitycited by 1
  36. 2024Inference Privacy: Properties and Mechanismscited by 0
  37. 2025L-Lipschitz Gershgorin ResNet Networkcited by 0
  38. 2025APECS: Adaptive Personalized Control System Architecturecited by 0

50 most cited of 58 All 58 (JSON)