# Regularizing deep networks using efficient layerwise adversarial training

arXiv: https://arxiv.org/abs/1705.07819 · HTML: https://mlsanity.com/paper/1705.07819/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2017.

## Built on by (most cited first, 24 of 24)

- [Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs](https://mlsanity.com/paper/2407.15549/) (2024)
- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Defending Against Unforeseen Failure Modes with Latent Adversarial Training](https://mlsanity.com/paper/2403.05030/) (2024)
- [Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities](https://mlsanity.com/paper/2502.05209/) (2025)
- [On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning](https://mlsanity.com/paper/2102.10454/) (2021)
- [Reliably fast adversarial training via latent adversarial perturbation](https://mlsanity.com/paper/2104.01575/) (2021)
- [Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective](https://mlsanity.com/paper/2404.19287/) (2024)
- [CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection](https://mlsanity.com/paper/2102.05311/) (2021)
- [Evaluating Similitude and Robustness of Deep Image Denoising Models via Adversarial Attack](https://mlsanity.com/paper/2306.16050/) (2023)
- [Adversarial Training: A Survey](https://mlsanity.com/paper/2410.15042/) (2024)
- [Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness](https://mlsanity.com/paper/2503.01345/) (2025)
- [Module-wise Adaptive Adversarial Training for End-to-end Autonomous Driving](https://mlsanity.com/paper/2409.07321/) (2024)
- [Adversarial Robustness in Unsupervised Machine Learning: A Systematic Review](https://mlsanity.com/paper/2306.00687/) (2023)
- [Adversarial Attacks and Defenses in Multivariate Time-Series Forecasting for Smart and Connected Infrastructures](https://mlsanity.com/paper/2408.14875/) (2024)
- [Layer-wise Regularized Adversarial Training using Layers Sustainability Analysis (LSA) framework](https://mlsanity.com/paper/2202.02626/) (2022)
- [CausAdv: A Causal-based Framework for Detecting Adversarial Examples](https://mlsanity.com/paper/2411.00839/) (2024)
- [Robust Regularization with Adversarial Labelling of Perturbed Samples](https://mlsanity.com/paper/2105.13745/) (2021)
- [Improving Adversarial Robustness for Free with Snapshot Ensemble](https://mlsanity.com/paper/2110.03124/) (2021)
- [Tracklet-Switch Adversarial Attack against Pedestrian Multi-Object Tracking Trackers](https://mlsanity.com/paper/2111.08954/) (2021)
- [Lateralized Learning for Multi-Class Visual Classification Tasks](https://mlsanity.com/paper/2301.12637/) (2023)
- [Defending Adversarial Examples by Negative Correlation Ensemble](https://mlsanity.com/paper/2206.10334/) (2022)
- [Adversarial Defenses via Vector Quantization](https://mlsanity.com/paper/2305.13651/) (2023)
- [Natias: Neuron Attribution based Transferable Image Adversarial Steganography](https://mlsanity.com/paper/2409.04968/) (2024)
- [MemLoss: Enhancing Adversarial Training with Recycling Adversarial Examples](https://mlsanity.com/paper/2510.09105/) (2025)

## Abstract (arXiv)

Adversarial training has been shown to regularize deep neural networks in addition to increasing their robustness to adversarial examples. However, its impact on very deep state of the art networks has not been fully investigated. In this paper, we present an efficient approach to perform adversarial training by perturbing intermediate layer activations and study the use of such perturbations as a regularizer during training. We use these perturbations to train very deep models such as ResNets and show improvement in performance both on adversarial and original test data. Our experiments highlight the benefits of perturbing intermediate layer activations compared to perturbing only the inputs. The results on CIFAR-10 and CIFAR-100 datasets show the merits of the proposed adversarial training approach. Additional results on WideResNets show that our approach provides significant improvement in classification accuracy for a given base model, outperforming dropout and other base models of larger size.