# A Unified Gradient Regularization Family for Adversarial Examples

arXiv: https://arxiv.org/abs/1511.06385 · HTML: https://mlsanity.com/paper/1511.06385/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2015.

## Built on by (most cited first, 30 of 43)

- [A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies](https://mlsanity.com/paper/2203.14046/) (2022)
- [The Geometry of Adversarial Training in Binary Classification](https://mlsanity.com/paper/2111.13613/) (2021)
- [Generative Dynamic Patch Attack](https://mlsanity.com/paper/2111.04266/) (2021)
- [Erasing, Transforming, and Noising Defense Network for Occluded Person Re-Identification](https://mlsanity.com/paper/2307.07187/) (2023)
- [Why Adversarial Reprogramming Works, When It Fails, and How to Tell the Difference](https://mlsanity.com/paper/2108.11673/) (2021)
- [Deep Learning for Brain Age Estimation: A Systematic Review](https://mlsanity.com/paper/2212.03868/) (2022)
- [Towards a multi-stakeholder value-based assessment framework for algorithmic systems](https://mlsanity.com/paper/2205.04525/) (2022)
- [Exact Count of Boundary Pieces of ReLU Classifiers: Towards the Proper Complexity Measure for Classification](https://mlsanity.com/paper/2306.08805/) (2023)
- [Towards a Robust and Trustworthy Machine Learning System Development: An Engineering Perspective](https://mlsanity.com/paper/2101.03042/) (2021)
- [Improving Model Robustness with Latent Distribution Locally and Globally](https://mlsanity.com/paper/2107.04401/) (2021)
- [On the regularized risk of distributionally robust learning over deep neural networks](https://mlsanity.com/paper/2109.06294/) (2021)
- [Nash Equilibria, Regularization and Computation in Optimal Transport-Based Distributionally Robust Optimization](https://mlsanity.com/paper/2303.03900/) (2023)
- [Boosting Adversarial Training via Fisher-Rao Norm-based Regularization](https://mlsanity.com/paper/2403.17520/) (2024)
- [We Can Always Catch You: Detecting Adversarial Patched Objects WITH or WITHOUT Signature](https://mlsanity.com/paper/2106.05261/) (2021)
- [Characterizing Model Robustness via Natural Input Gradients](https://mlsanity.com/paper/2409.20139/) (2024)
- [Bridging the Gap Between Adversarial Robustness and Optimization Bias](https://mlsanity.com/paper/2102.08868/) (2021)
- [Doubly Perturbed Task Free Continual Learning](https://mlsanity.com/paper/2312.13027/) (2023)
- [Deep neural networks for choice analysis: Enhancing behavioral regularity with gradient regularization](https://mlsanity.com/paper/2404.14701/) (2024)
- [On adversarial robustness and the use of Wasserstein ascent-descent dynamics to enforce it](https://mlsanity.com/paper/2301.03662/) (2023)
- [Improving Generalization via Uncertainty Driven Perturbations](https://mlsanity.com/paper/2202.05737/) (2022)
- [Hessian-Free Second-Order Adversarial Examples for Adversarial Learning](https://mlsanity.com/paper/2207.01396/) (2022)
- [Adversarially Robust Feature Learning for Breast Cancer Diagnosis](https://mlsanity.com/paper/2402.08768/) (2024)
- [Ensemble Defense with Data Diversity: Weak Correlation Implies Strong Robustness](https://mlsanity.com/paper/2106.02867/) (2021)
- [ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A Case Study](https://mlsanity.com/paper/2401.06513/) (2024)
- [SmoothGuard: Defending Multimodal Large Language Models with Noise Perturbation and Clustering Aggregation](https://mlsanity.com/paper/2510.26830/) (2025)
- [Evolving Architectures with Gradient Misalignment toward Low Adversarial Transferability](https://mlsanity.com/paper/2109.05919/) (2021)
- [Robustness and invariance properties of image classifiers](https://mlsanity.com/paper/2209.02408/) (2022)
- [Adversarial Amendment is the Only Force Capable of Transforming an Enemy into a Friend](https://mlsanity.com/paper/2305.10766/) (2023)
- [Adversarially robust generalization theory via Jacobian regularization for deep neural networks](https://mlsanity.com/paper/2412.12449/) (2024)
- [Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approach](https://mlsanity.com/paper/2507.22855/) (2025)

## Abstract (arXiv)

Adversarial examples are augmented data points generated by imperceptible perturbation of input samples. They have recently drawn much attention with the machine learning and data mining community. Being difficult to distinguish from real examples, such adversarial examples could change the prediction of many of the best learning models including the state-of-the-art deep learning models. Recent attempts have been made to build robust models that take into account adversarial examples. However, these methods can either lead to performance drops or lack mathematical motivations. In this paper, we propose a unified framework to build robust machine learning models against adversarial examples. More specifically, using the unified framework, we develop a family of gradient regularization methods that effectively penalize the gradient of loss function w.r.t. inputs. Our proposed framework is appealing in that it offers a unified view to deal with adversarial examples. It incorporates another recently-proposed perturbation based approach as a special case. In addition, we present some visual effects that reveals semantic meaning in those perturbations, and thus support our regularization method and provide another explanation for generalizability of adversarial examples. By applying this technique to Maxout networks, we conduct a series of experiments and achieve encouraging results on two benchmark datasets. In particular,we attain the best accuracy on MNIST data (without data augmentation) and competitive performance on CIFAR-10 data.