# Large Norms of CNN Layers Do Not Hurt Adversarial Robustness

arXiv: https://arxiv.org/abs/2009.08435 · HTML: https://mlsanity.com/paper/2009.08435/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2020.

## Built on by (most cited first, 7 of 7)

- [CLIP: Cheap Lipschitz Training of Neural Networks](https://mlsanity.com/paper/2103.12531/) (2021)
- [Convolutional Neural Networks as 2-D systems](https://mlsanity.com/paper/2303.03042/) (2023)
- [The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective](https://mlsanity.com/paper/2405.16918/) (2024)
- [A Black-Box Evaluation Framework for Semantic Robustness in Bird's Eye View Detection](https://mlsanity.com/paper/2412.13913/) (2024)
- [When Flatness Does (Not) Guarantee Adversarial Robustness](https://mlsanity.com/paper/2510.14231/) (2025)
- [A Tunable Despeckling Neural Network Stabilized via Diffusion Equation](https://mlsanity.com/paper/2411.15921/) (2024)
- [On the Properties of Adversarially-Trained CNNs](https://mlsanity.com/paper/2203.09243/) (2022)

## Abstract (arXiv)

Since the Lipschitz properties of convolutional neural networks (CNNs) are widely considered to be related to adversarial robustness, we theoretically characterize the ℓ1 norm and ℓ_∞ norm of 2D multi-channel convolutional layers and provide efficient methods to compute the exact ℓ1 norm and ℓ_∞ norm. Based on our theorem, we propose a novel regularization method termed norm decay, which can effectively reduce the norms of convolutional layers and fully-connected layers. Experiments show that norm-regularization methods, including norm decay, weight decay, and singular value clipping, can improve generalization of CNNs. However, they can slightly hurt adversarial robustness. Observing this unexpected phenomenon, we compute the norms of layers in the CNNs trained with three different adversarial training frameworks and surprisingly find that adversarially robust CNNs have comparable or even larger layer norms than their non-adversarially robust counterparts. Furthermore, we prove that under a mild assumption, adversarially robust classifiers can be achieved using neural networks, and an adversarially robust neural network can have an arbitrarily large Lipschitz constant. For this reason, enforcing small norms on CNN layers may be neither necessary nor effective in achieving adversarial robustness. The code is available at https://github.com/youweiliang/normrobustness.