# L2-Nonexpansive Neural Networks

arXiv: https://arxiv.org/abs/1802.07896 · HTML: https://mlsanity.com/paper/1802.07896/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Feb 2018.

## Built on by (most cited first, 21 of 21)

- [Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks](https://mlsanity.com/paper/2110.03825/) (2021)
- [Skew Orthogonal Convolutions](https://mlsanity.com/paper/2105.11417/) (2021)
- [Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds](https://mlsanity.com/paper/2111.01395/) (2021)
- [Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective](https://mlsanity.com/paper/2210.01787/) (2022)
- [Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100](https://mlsanity.com/paper/2108.04062/) (2021)
- [Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustness](https://mlsanity.com/paper/2104.05942/) (2021)
- [Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neurons](https://mlsanity.com/paper/2102.05363/) (2021)
- [LOT: Layer-wise Orthogonal Training on Improving ℓ2 Certified Robustness](https://mlsanity.com/paper/2210.11620/) (2022)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [Boosting the Certified Robustness of L-infinity Distance Nets](https://mlsanity.com/paper/2110.06850/) (2021)
- [Existence, Stability and Scalability of Orthogonal Convolutional Neural Networks](https://mlsanity.com/paper/2108.05623/) (2021)
- [Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training](https://mlsanity.com/paper/2103.00673/) (2021)
- [Learning Diverse-Structured Networks for Adversarial Robustness](https://mlsanity.com/paper/2102.01886/) (2021)
- [Improved techniques for deterministic l2 robustness](https://mlsanity.com/paper/2211.08453/) (2022)
- [CARTL: Cooperative Adversarially-Robust Transfer Learning](https://mlsanity.com/paper/2106.06667/) (2021)
- [Certified Defense via Latent Space Randomized Smoothing with Orthogonal Encoders](https://mlsanity.com/paper/2108.00491/) (2021)
- [Stabilizing Quantization-Aware Training by Implicit-Regularization on Hessian Matrix](https://mlsanity.com/paper/2503.11159/) (2025)
- [Depthwise Separable Convolutions Allow for Fast and Memory-Efficient Spectral Normalization](https://mlsanity.com/paper/2102.06496/) (2021)
- [Leveraging Local Variation in Data: Sampling and Weighting Schemes for Supervised Deep Learning](https://mlsanity.com/paper/2101.07561/) (2021)
- [An Ensemble Approach Towards Adversarial Robustness](https://mlsanity.com/paper/2106.05996/) (2021)
- [1-Lipschitz Neural Networks on Hadamard Manifolds](https://mlsanity.com/paper/2607.19335/) (2026)

## Abstract (arXiv)

This paper proposes a class of well-conditioned neural networks in which a unit amount of change in the inputs causes at most a unit amount of change in the outputs or any of the internal layers. We develop the known methodology of controlling Lipschitz constants to realize its full potential in maximizing robustness, with a new regularization scheme for linear layers, new ways to adapt nonlinearities and a new loss function. With MNIST and CIFAR-10 classifiers, we demonstrate a number of advantages. Without needing any adversarial training, the proposed classifiers exceed the state of the art in robustness against white-box L2-bounded adversarial attacks. They generalize better than ordinary networks from noisy data with partially random labels. Their outputs are quantitatively meaningful and indicate levels of confidence and generalization, among other desirable properties.