# Towards Deep Neural Network Architectures Robust to Adversarial Examples

arXiv: https://arxiv.org/abs/1412.5068 · HTML: https://mlsanity.com/paper/1412.5068/

## Key facts

- Cited by 146 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2014.

## Built on by (most cited first, 30 of 146)

- [Scaling Laws for Reward Model Overoptimization](https://mlsanity.com/paper/2210.10760/) (2022)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [On the Convergence and Robustness of Adversarial Training](https://mlsanity.com/paper/2112.08304/) (2021)
- [Improving Adversarial Robustness via Channel-wise Activation Suppressing](https://mlsanity.com/paper/2103.08307/) (2021)
- [How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses](https://mlsanity.com/paper/2305.10862/) (2023)
- [Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems](https://mlsanity.com/paper/2401.05778/) (2024)
- [Robust Semantic Communications with Masked VQ-VAE Enabled Codebook](https://mlsanity.com/paper/2206.04011/) (2022)
- [Adversarial Example Detection for DNN Models: A Review and Experimental Comparison](https://mlsanity.com/paper/2105.00203/) (2021)
- [Shape-invariant 3D Adversarial Point Clouds](https://mlsanity.com/paper/2203.04041/) (2022)
- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Online Adversarial Purification based on Self-Supervision](https://mlsanity.com/paper/2101.09387/) (2021)
- [Optical Adversarial Attack](https://mlsanity.com/paper/2108.06247/) (2021)
- [Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability](https://mlsanity.com/paper/2209.08025/) (2022)
- [Physical Adversarial Attacks on an Aerial Imagery Object Detector](https://mlsanity.com/paper/2108.11765/) (2021)
- [Immunization against harmful fine-tuning attacks](https://mlsanity.com/paper/2402.16382/) (2024)
- [On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Models](https://mlsanity.com/paper/2103.07101/) (2021)
- [Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses](https://mlsanity.com/paper/2104.01789/) (2021)
- [Nearly Optimal VC-Dimension and Pseudo-Dimension Bounds for Deep Neural Network Derivatives](https://mlsanity.com/paper/2305.08466/) (2023)
- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective](https://mlsanity.com/paper/2404.19287/) (2024)
- [reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputs](https://mlsanity.com/paper/2503.11751/) (2025)
- [Defending against Adversarial Audio via Diffusion Model](https://mlsanity.com/paper/2303.01507/) (2023)
- [Generative Dynamic Patch Attack](https://mlsanity.com/paper/2111.04266/) (2021)
- [Beyond Dropout: Robust Convolutional Neural Networks Based on Local Feature Masking](https://mlsanity.com/paper/2407.13646/) (2024)
- [Removing Adversarial Noise in Class Activation Feature Space](https://mlsanity.com/paper/2104.09197/) (2021)
- [On the Effectiveness of Small Input Noise for Defending Against Query-based Black-Box Attacks](https://mlsanity.com/paper/2101.04829/) (2021)
- [Towards Adversarially Robust Dataset Distillation by Curvature Regularization](https://mlsanity.com/paper/2403.10045/) (2024)
- [Detecting Adversarial Examples by Input Transformations, Defense Perturbations, and Voting](https://mlsanity.com/paper/2101.11466/) (2021)

## Abstract (arXiv)

Recent work has shown deep neural networks (DNNs) to be highly susceptible to well-designed, small perturbations at the input layer, or so-called adversarial examples. Taking images as an example, such distortions are often imperceptible, but can result in 100% mis-classification for a state of the art DNN. We study the structure of adversarial examples and explore network topology, pre-processing and training strategies to improve the robustness of DNNs. We perform various experiments to assess the removability of adversarial examples by corrupting with additional noise and pre-processing with denoising autoencoders (DAEs). We find that DAEs can remove substantial amounts of the adversarial noise. How- ever, when stacking the DAE with the original DNN, the resulting network can again be attacked by new adversarial examples with even smaller distortion. As a solution, we propose Deep Contractive Network, a model with a new end-to-end training procedure that includes a smoothness penalty inspired by the contractive autoencoder (CAE). This increases the network robustness to adversarial examples, without a significant performance penalty.