mlsanityData to 7 Oct 2026

Exactly Computing the Local Lipschitz Constant of ReLU Networks

arXiv 2003.01219 ↗First posted 2 Mar 2020stat.ML cs.LGcited by 102

Not on the map. Why?

Abstract (arXiv)

The local Lipschitz constant of a neural network is a useful metric with applications in robustness, generalization, and fairness evaluation. We provide novel analytic results relating the local Lipschitz constant of nonsmooth vector-valued functions to a maximization over the norm of the generalized Jacobian. We present a sufficient condition for which backpropagation always returns an element of the generalized Jacobian, and reframe the problem over this broad class of functions. We show strong inapproximability results for estimating Lipschitz constants of ReLU networks, and then formulate an algorithm to compute these quantities exactly. We leverage this algorithm to evaluate the tightness of competing Lipschitz estimators and the effects of regularized training on the Lipschitz constant.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 21; 2023: 20; 2024: 19; 2025: 20; 2026: 8.

102 papers.

Sort:

By area:

  1. 2022Learning Smooth Neural Functions via Lipschitz Regularizationcited by 86
  2. 2021Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Boundscited by 64
  3. 2022Contrastive Adapters for Foundation Model Group Robustnesscited by 62
  4. 2022Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagationcited by 45
  5. 2025On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learningcited by 42
  6. 2022On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learningcited by 39
  7. 2023When Deep Learning Meets Polyhedral Theory: A Surveycited by 39
  8. 2023Some Fundamental Aspects about Lipschitz Continuity of Neural Networkscited by 25
  9. 2021Measuring Generalization with Optimal Transportcited by 24
  10. 2021ANCER: Anisotropic Certification via Sample-wise Volume Maximizationcited by 23
  11. 2021Learning by Transference: Training Graph Neural Networks on Growing Graphscited by 22
  12. 2021Parameterizing Activation Functions for Adversarial Robustnesscited by 22
38 more
  1. 2023Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iterationcited by 19
  2. 2023Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularizationcited by 19
  3. 2023Improving the Accuracy-Robustness Trade-Off of Classifiers via Adaptive Smoothingcited by 18
  4. 2021LipBaB: Computing exact Lipschitz constant of ReLU networkscited by 16
  5. 2022A Quantitative Geometric Approach to Neural-Network Smoothnesscited by 16
  6. 2022Chordal Sparsity for Lipschitz Constant Estimation of Deep Neural Networkscited by 15
  7. 2021Model Error Propagation via Learned Contraction Metrics for Safe Feedback Motion Planning of Unknown Systemscited by 13
  8. 2021Reliably-stabilizing piecewise-affine neural network controllerscited by 13
  9. 2022Instance-Dependent Generalization Bounds via Optimal Transportcited by 13
  10. 2024Gradient-Regularized Out-of-Distribution Detectioncited by 13
  11. 2021Analytical bounds on the local Lipschitz constants of ReLU networkscited by 12
  12. 2021On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Trainingcited by 11
  13. 2022Spelunking the Deep: Guaranteed Queries on General Neural Implicit Surfaces via Range Analysiscited by 11
  14. 2022Adversarial robustness of sparse local Lipschitz predictorscited by 11
  15. 2022Learning-based Design of Luenberger Observers for Autonomous Nonlinear Systemscited by 11
  16. 2022Statistical Safety and Robustness Guarantees for Feedback Motion Planning of Unknown Underactuated Stochastic Systemscited by 11
  17. 2023On the Lipschitz Constant of Deep Networks and Double Descentcited by 11
  18. 2022Stability Verification of Neural Network Controllers using Mixed-Integer Programmingcited by 10
  19. 2023Injectivity of ReLU networks: perspectives from statistical physicscited by 10
  20. 2023Robust One-Class Classification with Signed Distance Function using 1-Lipschitz Neural Networkscited by 10
  21. 2024ECLipsE: Efficient Compositional Lipschitz Constant Estimation for Deep Neural Networkscited by 10
  22. 2024Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifierscited by 10
  23. 2025Safe exploration in reproducing kernel Hilbert spacescited by 10
  24. 2024Lipschitz constant estimation for general neural network architectures using control toolscited by 8
  25. 20241-Lipschitz Neural Distance Fieldscited by 8
  26. 2021Provable Lipschitz Certification for Generative Modelscited by 7
  27. 2022Memory-efficient model-based deep learning with convergence and robustness guaranteescited by 7
  28. 2022One-vs-the-Rest Loss to Focus on Important Samples in Adversarial Trainingcited by 7
  29. 2024Verification of Geometric Robustness of Neural Networks via Piecewise Linear Approximation and Lipschitz Optimisationcited by 7
  30. 2025The Computational Complexity of Counting Linear Regions in ReLU Neural Networkscited by 7
  31. 2023CertViT: Certified Robustness of Pre-Trained Vision Transformerscited by 6
  32. 2024Novel Quadratic Constraints for Extending LipSDP beyond Slope-Restricted Activationscited by 6
  33. 2021What training reveals about neural network complexitycited by 5
  34. 2021Adaptive Differentially Private Empirical Risk Minimizationcited by 5
  35. 2022Zonotope Domains for Lagrangian Neural Network Verificationcited by 5
  36. 2023A Certified Radius-Guided Attack Framework to Image Segmentation Modelscited by 5
  37. 2023Hybrid Zonotopes Exactly Represent ReLU Neural Networkscited by 4
  38. 2024Probabilistic Lipschitzness and the Stable Rank for Comparing Explanation Modelscited by 4

50 most cited of 102 All 102 (JSON)