mlsanityData to 7 Oct 2026

Spectral Norm Regularization for Improving the Generalizability of Deep Learning

arXiv 1705.10941 ↗First posted 31 May 2017stat.ML cs.LGcited by 178

Not on the map. Why?

Abstract (arXiv)

We investigate the generalizability of deep learning based on the sensitivity to input perturbation. We hypothesize that the high sensitivity to the perturbation of data degrades the performance on it. To reduce the sensitivity to perturbation, we propose a simple and effective regularization method, referred to as spectral norm regularization, which penalizes the high spectral norm of weight matrices in neural networks. We provide supportive evidence for the abovementioned hypothesis by experimentally confirming that the models trained using spectral norm regularization exhibit better generalizability than other baseline methods.

Key facts. Cited by 178 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 30; 2022: 28; 2023: 25; 2024: 31; 2025: 38; 2026: 26.

178 papers.

Sort:

By area:

  1. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 150
  2. 2021A Universal Law of Robustness via Isoperimetrycited by 150
  3. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  4. 2023Parameter-Efficient Orthogonal Finetuning via Butterfly Factorizationcited by 112
  5. 2021Hybrid Spectrogram and Waveform Source Separationcited by 90
  6. 2023Input Perturbation Reduces Exposure Bias in Diffusion Modelscited by 88
  7. 2022Learning Smooth Neural Functions via Lipschitz Regularizationcited by 86
  8. 2022Data-free Backdoor Removal based on Channel Lipschitznesscited by 77
  9. 2022PermutoSDF: Fast Multi-View Reconstruction with Implicit Surfaces using Permutohedral Latticescited by 72
  10. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 64
  11. 2022Contrastive Adapters for Foundation Model Group Robustnesscited by 62
  12. 2021A Dynamical System Perspective for Lipschitz Neural Networkscited by 59
38 more
  1. 2022Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspectivecited by 48
  2. 2021Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neuronscited by 40
  3. 2021RoMA: Robust Model Adaptation for Offline Model-based Optimizationcited by 39
  4. 2022On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learningcited by 39
  5. 2022Why neural networks find simple solutions: the many regularizers of geometric complexitycited by 38
  6. 2021Learning with Hyperspherical Uniformitycited by 35
  7. 2024AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Modelscited by 35
  8. 2024Log Neural Controlled Differential Equations: The Lie Brackets Make a Differencecited by 34
  9. 2024Learning Continually by Spectral Regularizationcited by 34
  10. 2023LipsFormer: Introducing Lipschitz Continuity to Vision Transformerscited by 33
  11. 2023Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferabilitycited by 32
  12. 2021Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networkscited by 30
  13. 2021Noise Stability Regularization for Improving BERT Fine-tuningcited by 29
  14. 2024Weight Clipping for Deep Continual and Reinforcement Learningcited by 29
  15. 2025RL Fine-Tuning Heals OOD Forgetting in SFTcited by 27
  16. 2021Boosting the Certified Robustness of L-infinity Distance Netscited by 24
  17. 2022Random matrix analysis of deep neural network weight matricescited by 21
  18. 2024How Do Recommendation Models Amplify Popularity Bias? An Analysis from the Spectral Perspectivecited by 21
  19. 2023Improving Masked Autoencoders by Learning Where to Maskcited by 20
  20. 2021Lipschitz Continuity Guided Knowledge Distillationcited by 19
  21. 2023Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iterationcited by 19
  22. 2023Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Trainingcited by 19
  23. 2023On the Robustness of Removal-Based Feature Attributionscited by 18
  24. 2025Training Transformers with Enforced Lipschitz Constantscited by 18
  25. 2024Zero-Shot Class Unlearning in CLIP with Synthetic Samplescited by 17
  26. 2021Learning Prediction Intervals for Regression: Generalization and Calibrationcited by 16
  27. 2022Lipschitz Continuity Retained Binary Neural Networkcited by 14
  28. 2022Improving GANs for Long-Tailed Data through Group Spectral Regularizationcited by 14
  29. 2023Transforming Radiance Field with Lipschitz Network for Photorealistic 3D Scene Stylizationcited by 14
  30. 2023On genuine invariance learning without weight-tyingcited by 13
  31. 2022Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularizationcited by 12
  32. 2023FairGAT: Fairness-aware Graph Attention Networkscited by 12
  33. 2023FaceCLIPNeRF: Text-driven 3D Face Manipulation using Deformable Neural Radiance Fieldscited by 12
  34. 2023DP-SGD Without Clipping: The Lipschitz Neural Network Waycited by 11
  35. 2024ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networkscited by 11
  36. 2024Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivitycited by 11
  37. 2024Mitigating Memorization In Language Modelscited by 11
  38. 2024Towards Stabilized and Efficient Diffusion Transformers through Long-Skip-Connections with Spectral Constraintscited by 11

50 most cited of 178 All 178 (JSON)