mlsanityData to 7 Oct 2026

Regularisation of Neural Networks by Enforcing Lipschitz Continuity

arXiv 1804.04368 ↗First posted 12 Apr 2018stat.ML cs.LGcited by 311

Not on the map. Why?

Abstract (arXiv)

We investigate the effect of explicitly enforcing the Lipschitz continuity of neural networks with respect to their inputs. To this end, we provide a simple technique for computing an upper bound to the Lipschitz constant---for multiple p-norms---of a feed forward neural network composed of commonly used layer types. Our technique is then used to formulate training a neural network with a bounded Lipschitz constant as a constrained optimisation problem that can be solved using projected stochastic gradient methods. Our evaluation study shows that the performance of the resulting models exceeds that of models trained with other common regularisers. We also provide evidence that the hyperparameters are intuitive to tune, demonstrate how the choice of norm for computing the Lipschitz constant impacts the resulting model, and show that the performance gains provided by our method are particularly noticeable when only a small amount of training data is available.

Key facts. Cited by 311 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 57; 2022: 48; 2023: 51; 2024: 60; 2025: 55; 2026: 39.

311 papers.

Sort:

By area:

  1. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  2. 2021Machine Unlearning of Features and Labelscited by 223
  3. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  4. 2022AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Timecited by 199
  5. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Datacited by 171
  6. 2021Towards a Unified Framework for Fair and Stable Graph Representation Learningcited by 128
  7. 2021Globally-Robust Neural Networkscited by 106
  8. 2021Orthogonalizing Convolutional Layers with the Cayley Transformcited by 105
  9. 2021Neural Flows: Efficient Alternative to Neural ODEscited by 99
  10. 2021On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertaintycited by 87
  11. 2022Learning Smooth Neural Functions via Lipschitz Regularizationcited by 86
  12. 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 84
38 more
  1. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
  2. 2021Skew Orthogonal Convolutionscited by 67
  3. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 64
  4. 2022Contrastive Adapters for Foundation Model Group Robustnesscited by 62
  5. 2021A Dynamical System Perspective for Lipschitz Neural Networkscited by 59
  6. 2021Exploring Counterfactual Explanations Through the Lens of Adversarial Examples: A Theoretical and Empirical Analysiscited by 52
  7. 2022Rethinking Stability for Attribution-based Explanationscited by 52
  8. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  9. 2023Direct Parameterization of Lipschitz-Bounded Deep Networkscited by 49
  10. 2022Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspectivecited by 48
  11. 2021Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100cited by 47
  12. 2022Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagationcited by 45
  13. 2021Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustnesscited by 43
  14. 2021Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neuronscited by 40
  15. 2022Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networkscited by 39
  16. 2022On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learningcited by 39
  17. 2021Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networkscited by 38
  18. 2021Probing GNN Explainers: A Rigorous Theoretical and Empirical Analysis of GNN Explanation Methodscited by 36
  19. 2021Neural Network Robustness as a Verification Property: A Principled Case Studycited by 35
  20. 2023Policy Regularization with Dataset Constraint for Offline Reinforcement Learningcited by 33
  21. 2022LOT: Layer-wise Orthogonal Training on Improving ℓ2 Certified Robustnesscited by 32
  22. 2021Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networkscited by 30
  23. 2021Low Curvature Activations Reduce Overfitting in Adversarial Trainingcited by 29
  24. 2021CLIP: Cheap Lipschitz Training of Neural Networkscited by 29
  25. 2024Weight Clipping for Deep Continual and Reinforcement Learningcited by 29
  26. 2021Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Datacited by 27
  27. 2024Learning Smooth Humanoid Locomotion through Lipschitz-Constrained Policiescited by 26
  28. 2023Some Fundamental Aspects about Lipschitz Continuity of Neural Networkscited by 25
  29. 2021Boosting the Certified Robustness of L-infinity Distance Netscited by 24
  30. 2022Robustness Implies Generalization via Data-Dependent Generalization Boundscited by 23
  31. 2024Training Free Guided Flow Matching with Optimal Controlcited by 23
  32. 2021Reliable Graph Neural Networks for Drug Discovery Under Distributional Shiftcited by 22
  33. 2022L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learningcited by 22
  34. 2023Reducing Communication for Split Learning by Randomized Top-k Sparsificationcited by 22
  35. 2024Uncertainty Estimation by Density Aware Evidential Deep Learningcited by 22
  36. 2023Training robust and generalizable quantum modelscited by 21
  37. 2024H-Consistency Guarantees for Regressioncited by 21
  38. 2024Doubly Mild Generalization for Offline Reinforcement Learningcited by 21

50 most cited of 311 All 311 (JSON)