mlsanityData to 14 Oct 2026

Exact Spectral Norm Regularization for Neural Networks

arXiv 2206.13581 ↗First posted 27 Jun 2022stat.ML cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We pursue a line of research that seeks to regularize the spectral norm of the Jacobian of the input-output mapping for deep neural networks. While previous work rely on upper bounding techniques, we provide a scheme that targets the exact spectral norm. We showcase that our algorithm achieves an improved generalization performance compared to previous spectral regularization techniques while simultaneously maintaining a strong safeguard against natural and adversarial noise. Moreover, we further explore some previous reasoning concerning the strong adversarial protection that Jacobian regularization provides and show that it can be misleading.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 20 of its 30 references that are in the map. First posted on arXiv 27 Jun 2022.

Builds on

20 of 30 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  7. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  8. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  9. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  10. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 678
  11. 2018Deep Learning for Classical Japanese Literaturecited by 422
  12. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 182
8 more
  1. 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 161
  2. 2018ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detectorcited by 159
  3. 2020Torchattacks: A PyTorch Repository for Adversarial Attackscited by 158
  4. 2014Towards Deep Neural Network Architectures Robust to Adversarial Examplescited by 146
  5. 2019Robust Learning with Jacobian Regularizationcited by 129
  6. 2016Robust Large Margin Deep Neural Networkscited by 118
  7. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 65
  8. 2019Fooling a Real Car with Adversarial Traffic Signscited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 1; 2026: 2.

5 papers.

  1. 2022Generalizing and Improving Jacobian and Hessian Regularizationcited by 2
  2. 2025CAST: Compositional Analysis via Spectral Tracking for Understanding Transformer Layer Functionscited by 2
  3. 2023Flat Minima in Linear Estimation and an Extended Gauss Markov Theoremcited by 1
  4. 2026Theory of Minimal Weight Perturbations in Deep Networks and its Applications for Low-Rank Activated Backdoor Attackscited by 1
  5. 2026Dynamics to decision: A mathematical theory of Lyapunov spectra and decision boundaries in deep classifierscited by 0