mlsanityData to 7 Oct 2026

Gradient Regularization Improves Accuracy of Discriminative Models

arXiv 1712.09936 ↗First posted 28 Dec 2017cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

Regularizing the gradient norm of the output of a neural network with respect to its inputs is a powerful technique, rediscovered several times. This paper presents evidence that gradient regularization can consistently improve classification accuracy on vision tasks, using modern deep neural networks, especially when the amount of training data is small. We introduce our regularizers as members of a broader class of Jacobian-based regularizers. We demonstrate empirically on real and synthetic data that the learning process leads to gradients controlled beyond the training points, and results in solutions that generalize well.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 6; 2023: 4; 2024: 8; 2025: 0; 2026: 3.

26 papers.

Sort:

  1. 2022Learning Smooth Neural Functions via Lipschitz Regularizationcited by 86
  2. 2021On Linear Stability of SGD and Input-Smoothness of Neural Networkscited by 61
  3. 2022Why neural networks find simple solutions: the many regularizers of geometric complexitycited by 38
  4. 2023Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferabilitycited by 32
  5. 2021Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularizationcited by 17
  6. 2024PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularizationcited by 14
  7. 2021Why Adversarial Reprogramming Works, When It Fails, and How to Tell the Differencecited by 13
  8. 2024Nuclear Norm Regularization for Deep Learningcited by 11
  9. 2024PAC-Bayes-Chernoff bounds for unbounded lossescited by 9
  10. 2023Nash Equilibria, Regularization and Computation in Optimal Transport-Based Distributionally Robust Optimizationcited by 8
  11. 2021The Geometric Occam's Razor Implicit in Deep Learningcited by 7
  12. 2021Jacobian Regularization for Mitigating Universal Adversarial Perturbationscited by 4
14 more
  1. 2024MIRACLE3D: Memory-efficient Integrated Robust Approach for Continual Learning on Point Clouds via Shape Model Constructioncited by 3
  2. 2022Training-Free Robust Multimodal Learning via Sample-Wise Jacobian Regularizationcited by 2
  3. 2022Machines of finite depth: towards a formalization of neural networkscited by 2
  4. 2022Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separationcited by 2
  5. 2022Generalizing and Improving Jacobian and Hessian Regularizationcited by 2
  6. 2024From Robustness to Improved Generalization and Calibration in Pre-trained Language Modelscited by 1
  7. 2024Universal Robustness via Median Randomized Smoothing for Real-World Super-Resolutioncited by 1
  8. 2024Adversarially robust generalization theory via Jacobian regularization for deep neural networkscited by 1
  9. 2026Smoothness-Based Derandomization of PAC-Bayes Boundscited by 1
  10. 2023Unsupervised Learning of Initialization in Deep Neural Networks via Maximum Mean Discrepancycited by 0
  11. 2023On Continuity of Robust and Accurate Classifierscited by 0
  12. 2024Exploring Biologically Inspired Mechanisms of Adversarial Robustnesscited by 0
  13. 2026CMR: Contractive Mapping Embeddings for Robust Humanoid Locomotion on Unstructured Terrainscited by 0
  14. 2026SaliencyDecor: Enhancing Neural Network Interpretability through Feature Decorrelationcited by 0