mlsanityData to 14 Oct 2026

Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration

arXiv 2305.16173 ↗First posted 25 May 2023cs.LG cs.AIcited by 19

Not on the map. Why?

Abstract (arXiv)

Since the control of the Lipschitz constant has a great impact on the training stability, generalization, and robustness of neural networks, the estimation of this value is nowadays a real scientific challenge. In this paper we introduce a precise, fast, and differentiable upper bound for the spectral norm of convolutional layers using circulant matrix theory and a new alternative to the Power iteration. Called the Gram iteration, our approach exhibits a superlinear convergence. First, we show through a comprehensive set of experiments that our approach outperforms other state-of-the-art methods in terms of precision, computational cost, and scalability. Then, it proves highly effective for the Lipschitz regularization of convolutional neural networks, with competitive results against concurrent approaches. Code is available at https://github.com/blaisedelattre/lip4conv.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

25 of 38 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  4. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  5. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 415
  6. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 314
  7. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
  8. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 293
  9. 2018Sorting out Lipschitz function approximationcited by 213
  10. 2019Plug-and-Play Methods Provably Converge with Properly Trained Denoiserscited by 198
  11. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 195
  12. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 182
13 more
  1. 2019Orthogonal Convolutional Neural Networkscited by 123
  2. 2021Orthogonalizing Convolutional Layers with the Cayley Transformcited by 105
  3. 2020Exactly Computing the Local Lipschitz Constant of ReLU Networkscited by 102
  4. 2018The Singular Values of Convolutional Layerscited by 95
  5. 2019Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networkscited by 87
  6. 2020Lipschitz constant estimation of Neural Networks via sparse polynomial optimizationcited by 87
  7. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  8. 2022Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspectivecited by 51
  9. 2022Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagationcited by 45
  10. 2020Controllable Orthogonalization in Training DNNscited by 30
  11. 2018RecurJac: An Efficient Recursive Algorithm for Bounding Jacobian Matrix of Neural Networks and Its Applicationscited by 27
  12. 2018Bounding Multivariate Trigonometric Polynomials with Applications to Filter Bank Designcited by 5
  13. 2020Asymptotic Singular Value Distribution of Linear Convolutional Layerscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 6; 2026: 6.

19 papers.

Sort:

By area:

  1. 2023The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothingcited by 8
  2. 2025An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architecturescited by 8
  3. 2024Spectrum Extraction and Clipping for Implicitly Linear Layerscited by 7
  4. 2024Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layerscited by 7
  5. 2024Spectral Norm of Convolutional Layers with Circular and Zero Paddingscited by 4
  6. 2023Certified Robust Models with Slack Control and Large Lipschitz Constantscited by 3
  7. 2025Improved Scalable Lipschitz Bounds for Deep Neural Networkscited by 2
  8. 2025Bridging the Theoretical Gap in Randomized Smoothingcited by 2
  9. 2024LOTOS: Layer-wise Orthogonalization for Training Robust Ensemblescited by 1
  10. 2025A Formally Verified Robustness Certifier for Neural Networks (Extended Version)cited by 1
  11. 2025ATOM: A Pretrained Neural Operator for Multitask Molecular Dynamicscited by 1
  12. 2026Cold-Start Forecasting of New Product Life-Cycles via Conditional Diffusion Modelscited by 1
7 more
  1. 2024HPPP: Halpern-type Preconditioned Proximal Point Algorithms and Applications to Image Restorationcited by 0
  2. 2025Beyond Wave Variables: A Data-Driven Ensemble Approach for Enhanced Teleoperation Transparency and Stabilitycited by 0
  3. 2026Orthogonium : A Unified, Efficient Library of Orthogonal and 1-Lipschitz Building Blockscited by 0
  4. 2026Deep Neural Networks as Iterated Function Systems and a Generalization Boundcited by 0
  5. 2026Lipschitz-Based Robustness Certification Under Floating-Point Executioncited by 0
  6. 2026From SGD to Muon: Adaptive Optimization via Schatten-p Normscited by 0
  7. 2026Scalable Incremental Robustness Analysis of Neural Network Feedback Systemscited by 0