# Deep Defense: Training DNNs with Improved Adversarial Robustness

arXiv: https://arxiv.org/abs/1803.00404 · HTML: https://mlsanity.com/paper/1803.00404/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2018.

## Built on by (most cited first, 30 of 42)

- [On the Convergence and Robustness of Adversarial Training](https://mlsanity.com/paper/2112.08304/) (2021)
- [A Comprehensive Evaluation Framework for Deep Model Robustness](https://mlsanity.com/paper/2101.09617/) (2021)
- [Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability](https://mlsanity.com/paper/2209.08025/) (2022)
- [Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses](https://mlsanity.com/paper/2104.01789/) (2021)
- [Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularization](https://mlsanity.com/paper/2310.00116/) (2023)
- [Revisiting Hilbert-Schmidt Information Bottleneck for Adversarial Robustness](https://mlsanity.com/paper/2106.02734/) (2021)
- [Pruning Adversarially Robust Neural Networks without Adversarial Examples](https://mlsanity.com/paper/2210.04311/) (2022)
- [Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS](https://mlsanity.com/paper/2102.07244/) (2021)
- [AdvFilter: Predictive Perturbation-aware Filtering against Adversarial Attack via Multi-domain Learning](https://mlsanity.com/paper/2107.06501/) (2021)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labels](https://mlsanity.com/paper/2105.14676/) (2021)
- [Perceptually Constrained Adversarial Attacks](https://mlsanity.com/paper/2102.07140/) (2021)
- [Recent Advances in Large Margin Learning](https://mlsanity.com/paper/2103.13598/) (2021)
- [Selection of Source Images Heavily Influences the Effectiveness of Adversarial Attacks](https://mlsanity.com/paper/2106.07141/) (2021)
- [Explaining Adversarial Vulnerability with a Data Sparsity Hypothesis](https://mlsanity.com/paper/2103.00778/) (2021)
- [Adversarial Training for Gradient Descent: Analysis Through its Continuous-time Approximation](https://mlsanity.com/paper/2105.08037/) (2021)
- [SafeLight: Enhancing Security in Optical Convolutional Neural Network Accelerators](https://mlsanity.com/paper/2411.16712/) (2024)
- [Adaptive Feature Alignment for Adversarial Training](https://mlsanity.com/paper/2105.15157/) (2021)
- [Clustering Effect of (Linearized) Adversarial Robust Models](https://mlsanity.com/paper/2111.12922/) (2021)
- [An Eye for an Eye: Defending against Gradient-based Attacks with Gradients](https://mlsanity.com/paper/2202.01117/) (2022)
- [Adversarial Attack and Defense for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2202.03077/) (2022)
- [Deadwooding: Robust Global Pruning for Deep Neural Networks](https://mlsanity.com/paper/2202.05226/) (2022)
- [SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learning](https://mlsanity.com/paper/2204.00734/) (2022)
- [RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation](https://mlsanity.com/paper/2301.08092/) (2023)
- [Robust Upper Bounds for Adversarial Training](https://mlsanity.com/paper/2112.09279/) (2021)
- [Two Heads are Better than One: Robust Learning Meets Multi-branch Models](https://mlsanity.com/paper/2208.08083/) (2022)
- [Provably Safeguarding a Classifier from OOD and Adversarial Samples: an Extreme Value Theory Approach](https://mlsanity.com/paper/2501.10202/) (2025)
- [A Survey of Adversarial Defenses in Vision-based Systems: Categorization, Methods and Challenges](https://mlsanity.com/paper/2503.00384/) (2025)
- [Adversarial Examples in Environment Perception for Automated Driving (Review)](https://mlsanity.com/paper/2504.08414/) (2025)
- [Trustworthy Quantum Machine Learning: A Roadmap for Reliability, Robustness, and Security in the NISQ Era](https://mlsanity.com/paper/2511.02602/) (2025)

## Abstract (arXiv)

Despite the efficacy on a variety of computer vision tasks, deep neural networks (DNNs) are vulnerable to adversarial attacks, limiting their applications in security-critical systems. Recent works have shown the possibility of generating imperceptibly perturbed image inputs (a.k.a., adversarial examples) to fool well-trained DNN classifiers into making arbitrary predictions. To address this problem, we propose a training recipe named "deep defense". Our core idea is to integrate an adversarial perturbation-based regularizer into the classification objective, such that the obtained models learn to resist potential attacks, directly and precisely. The whole optimization problem is solved just like training a recursive network. Experimental results demonstrate that our method outperforms training with adversarial/Parseval regularizations by large margins on various datasets (including MNIST, CIFAR-10 and ImageNet) and different DNN architectures. Code and models for reproducing our results are available at https://github.com/ZiangYan/deepdefense.pytorch