mlsanityData to 7 Oct 2026

CLIP: Cheap Lipschitz Training of Neural Networks

arXiv 2103.12531 ↗First posted 23 Mar 2021cs.LG math.OC stat.MLcited by 29

Not on the map. Why?

Abstract (arXiv)

Despite the large success of deep neural networks (DNN) in recent years, most neural networks still lack mathematical guarantees in terms of stability. For instance, DNNs are vulnerable to small or even imperceptible input perturbations, so called adversarial examples, that can cause false predictions. This instability can have severe consequences in applications which influence the health and safety of humans, e.g., biomedical imaging or autonomous driving. While bounding the Lipschitz constant of a neural network improves stability, most methods rely on restricting the Lipschitz constants of each layer which gives a poor bound for the actual Lipschitz constant. In this paper we investigate a variational regularization method named CLIP for controlling the Lipschitz constant of a neural network, which can easily be integrated into the training procedure. We mathematically analyze the proposed model, in particular discussing the impact of the chosen regularization parameter on the output of the network. Finally, we numerically evaluate our method on both a nonlinear regression problem and the MNIST and Fashion-MNIST classification databases, and compare our results with a weight regularization approach.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 23 Mar 2021.

Builds on

22 of 30 references.

References · 22

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  6. 2016An overview of gradient descent optimization algorithmscited by 1,021
  7. 2019Adversarial Training for Free!cited by 638
  8. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  9. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
  10. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  11. 2018Sorting out Lipschitz function approximationcited by 211
  12. 2019Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operatorscited by 54
10 more
  1. 2018Limitations of the Lipschitz constant as a defense against adversarial examplescited by 48
  2. 2018On Lipschitz Bounds of General Convolutional Neural Networkscited by 29
  3. 2019Adversarial Lipschitz Regularizationcited by 29
  4. 2020Deep Neural Networks with Trainable Activations and Controlled Lipschitz Constantcited by 25
  5. 2018Lipschitz regularized Deep Neural Networks generalize and are adversarially robustcited by 19
  6. 2020Lipschitz Bounds and Provably Robust Training by Laplacian Smoothingcited by 17
  7. 2020Large Norms of CNN Layers Do Not Hurt Adversarial Robustnesscited by 7
  8. 2020Variational regularisation for inverse problems with imperfect forward operators and general noise modelscited by 6
  9. 2018Solution Paths of Variational Regularization Methods for Inverse Problemscited by 4
  10. 2020Towards Rapid and Robust Adversarial Training with One-Step Attackscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 6; 2023: 5; 2024: 6; 2025: 5; 2026: 2.

29 papers.

Sort:

By area:

  1. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 88
  2. 2021Exploring Misclassifications of Robust Neural Networks to Enhance Adversarial Attackscited by 33
  3. 2023Some Fundamental Aspects about Lipschitz Continuity of Neural Networkscited by 25
  4. 2022Improving Lipschitz-Constrained Neural Networks by Learning Activation Functionscited by 23
  5. 2021The Geometry of Adversarial Training in Binary Classificationcited by 20
  6. 2022A Quantitative Geometric Approach to Neural-Network Smoothnesscited by 16
  7. 2022Improving Robustness against Real-World and Worst-Case Distribution Shifts through Decision Region Quantificationcited by 16
  8. 2022Approximation of Lipschitz Functions using Deep Spline Neural Networkscited by 15
  9. 2021Connections between Numerical Algorithms for PDEs and Neural Networkscited by 10
  10. 2022Memory-efficient model-based deep learning with convergence and robustness guaranteescited by 7
  11. 2023Raising the Bar for Certified Adversarial Robustness with Diffusion Modelscited by 7
  12. 2026Approximation Theory for Lipschitz Continuous Transformerscited by 5
17 more
  1. 2022Learning Globally Smooth Functions on Manifoldscited by 4
  2. 2021Designing Rotationally Invariant Neural Networks from PDEs and Variational Methodscited by 3
  3. 2024Consistency of Neural Causal Partial Identificationcited by 3
  4. 2024Policy Verification in Stochastic Dynamical Systems Using Logarithmic Neural Certificatescited by 3
  5. 2024Invertible ResNets for Inverse Imaging Problems: Competitive Performance with Provable Regularization Propertiescited by 2
  6. 2024A Tunable Despeckling Neural Network Stabilized via Diffusion Equationcited by 2
  7. 2025Effective Data Pruning through Score Extrapolationcited by 2
  8. 2026Error Analysis of Bayesian Inverse Problems with Generative Priorscited by 2
  9. 2023Local monotone operator learning using non-monotone operators: MnM-MOLcited by 1
  10. 2025Consensus-based optimization for closed-box adversarial attacks and a connection to evolution strategiescited by 1
  11. 2021Learning Lipschitz Feedback Policies from Expert Demonstrations: Closed-Loop Guarantees, Generalization and Robustnesscited by 0
  12. 2023Plug-and-Play Deep Energy Model for Inverse problemscited by 0
  13. 2023Accelerated parallel MRI using memory efficient and robust monotone operator learning (MOL)cited by 0
  14. 2024Your Network May Need to Be Rewritten: Network Adversarial Based on High-Dimensional Function Graph Decompositioncited by 0
  15. 2025MAP Image Recovery with Guarantees using Locally Convex Multi-Scale Energy (LC-MUSE) Modelcited by 0
  16. 2025Deep End-to-End Posterior ENergy (DEEPEN) for image recoverycited by 0
  17. 2025Stable neural networks and connections to continuous dynamical systemscited by 0