mlsanityData to 7 Oct 2026

Analytical bounds on the local Lipschitz constants of ReLU networks

arXiv 2104.14672 ↗First posted 29 Apr 2021cs.LG stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

In this paper, we determine analytical upper bounds on the local Lipschitz constants of feedforward neural networks with ReLU activation functions. We do so by deriving Lipschitz constants and bounds for ReLU, affine-ReLU, and max pooling functions, and combining the results to determine a network-wide bound. Our method uses several insights to obtain tight bounds, such as keeping track of the zero elements of each layer, and analyzing the composition of affine and ReLU functions. Furthermore, we employ a careful computational approach which allows us to apply our method to large networks such as AlexNet and VGG-16. We present several examples using different networks, which show how our local Lipschitz bounds are tighter than the global Lipschitz bounds. We also show how our method can be applied to provide adversarial bounds for classification networks. These results show that our method produces the largest known bounds on minimum adversarial perturbations for large networks such as AlexNet and VGG-16.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 29 Apr 2021.

Builds on

16 of 25 references.

References · 16

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  5. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  6. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
  7. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  8. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
  9. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 193
  10. 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 187
  11. 2016Robust Large Margin Deep Neural Networkscited by 117
  12. 2020Exactly Computing the Local Lipschitz Constant of ReLU Networkscited by 102
4 more
  1. 2020Lipschitz constant estimation of Neural Networks via sparse polynomial optimizationcited by 87
  2. 2018On Lipschitz Bounds of General Convolutional Neural Networkscited by 29
  3. 2019Adversarial Lipschitz Regularizationcited by 29
  4. 2018Singular Values for ReLU Layerscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 0; 2024: 4; 2025: 3; 2026: 2.

12 papers.

  1. 2022Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagationcited by 45
  2. 2022Chordal Sparsity for Lipschitz Constant Estimation of Deep Neural Networkscited by 15
  3. 2024Intriguing Equivalence Structures of the Embedding Space of Vision Transformerscited by 8
  4. 2024Probabilistic Lipschitzness and the Stable Rank for Comparing Explanation Modelscited by 4
  5. 2022An Optimal Time Variable Learning Framework for Deep Neural Networkscited by 3
  6. 2024Policy Verification in Stochastic Dynamical Systems Using Logarithmic Neural Certificatescited by 3
  7. 2024Deep deterministic policy gradient with symmetric data augmentation for lateral attitude tracking control of a fixed-wing aircraftcited by 1
  8. 2025Stochastic Sample Approximations of (Local) Moduli of Continuitycited by 1
  9. 2025ECLipsE-Gen-Local: Efficient Compositional Local Lipschitz Estimates for Deep Neural Networkscited by 1
  10. 2025Robustness and Regularization in Hierarchical Re-Basincited by 1
  11. 2026Improving Adversarial Robustness of Attribution via Implicit Regularizationcited by 0
  12. 2026LipSSD: Lipschitz-Constrained Single-Shot Detection for Adversarially Robust Object Detectioncited by 0