# Harnessing the Vulnerability of Latent Layers in Adversarially Trained Models

arXiv: https://arxiv.org/abs/1905.05186 · HTML: https://mlsanity.com/paper/1905.05186/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2019.

## Built on by (most cited first, 26 of 26)

- [Rethinking Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2402.08787/) (2024)
- [Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs](https://mlsanity.com/paper/2407.15549/) (2024)
- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Defending Against Unforeseen Failure Modes with Latent Adversarial Training](https://mlsanity.com/paper/2403.05030/) (2024)
- [Improving Adversarial Robustness via Channel-wise Activation Suppressing](https://mlsanity.com/paper/2103.08307/) (2021)
- [How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses](https://mlsanity.com/paper/2305.10862/) (2023)
- [Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities](https://mlsanity.com/paper/2502.05209/) (2025)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [Reliably fast adversarial training via latent adversarial perturbation](https://mlsanity.com/paper/2104.01575/) (2021)
- [LAFEAT: Piercing Through Adversarial Defenses with Latent Features](https://mlsanity.com/paper/2104.09284/) (2021)
- [On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training](https://mlsanity.com/paper/2112.07324/) (2021)
- [Game-Theoretic Defenses for Robust Conformal Prediction Against Adversarial Attacks in Medical Imaging](https://mlsanity.com/paper/2411.04376/) (2024)
- [Improving Model Robustness with Latent Distribution Locally and Globally](https://mlsanity.com/paper/2107.04401/) (2021)
- [MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps](https://mlsanity.com/paper/2111.05073/) (2021)
- [TextDefense: Adversarial Text Detection based on Word Importance Entropy](https://mlsanity.com/paper/2302.05892/) (2023)
- [Adversarial Finetuning with Latent Representation Constraint to Mitigate Accuracy-Robustness Tradeoff](https://mlsanity.com/paper/2308.16454/) (2023)
- [Robust Binary Models by Pruning Randomly-initialized Networks](https://mlsanity.com/paper/2202.01341/) (2022)
- [Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided Curriculum Learning Approach](https://mlsanity.com/paper/2111.00295/) (2021)
- [The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective](https://mlsanity.com/paper/2405.16918/) (2024)
- [Beyond Vulnerabilities: A Survey of Adversarial Attacks as Both Threats and Defenses in Computer Vision Systems](https://mlsanity.com/paper/2508.01845/) (2025)
- [When Flatness Does (Not) Guarantee Adversarial Robustness](https://mlsanity.com/paper/2510.14231/) (2025)
- [Consistent Valid Physically-Realizable Adversarial Attack against Crowd-flow Prediction Models](https://mlsanity.com/paper/2303.02669/) (2023)
- [IRAD: Implicit Representation-driven Image Resampling against Adversarial Attacks](https://mlsanity.com/paper/2310.11890/) (2023)
- [A comment on Guo et al. [arXiv:2206.11228]](https://mlsanity.com/paper/2208.01456/) (2022)
- [A Survey of Adversarial CAPTCHAs on its History, Classification and Generation](https://mlsanity.com/paper/2311.13233/) (2023)
- [Robust Network Learning via Inverse Scale Variational Sparsification](https://mlsanity.com/paper/2409.18419/) (2024)

## Abstract (arXiv)

Neural networks are vulnerable to adversarial attacks -- small visually imperceptible crafted noise which when added to the input drastically changes the output. The most effective method of defending against these adversarial attacks is to use the methodology of adversarial training. We analyze the adversarially trained robust models to study their vulnerability against adversarial attacks at the level of the latent layers. Our analysis reveals that contrary to the input layer which is robust to adversarial attack, the latent layer of these robust models are highly susceptible to adversarial perturbations of small magnitude. Leveraging this information, we introduce a new technique Latent Adversarial Training (LAT) which comprises of fine-tuning the adversarially trained models to ensure the robustness at the feature layers. We also propose Latent Attack (LA), a novel algorithm for construction of adversarial examples. LAT results in minor improvement in test accuracy and leads to a state-of-the-art adversarial accuracy against the universal first-order adversarial PGD attack which is shown for the MNIST, CIFAR-10, CIFAR-100 datasets.