# Robust Feature-Level Adversaries are Interpretability Tools

arXiv: https://arxiv.org/abs/2110.03605 · HTML: https://mlsanity.com/paper/2110.03605/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 85 references that are in the map.
- First posted on arXiv 7 Oct 2021.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Do Adversarially Robust ImageNet Models Transfer Better?](https://mlsanity.com/paper/2007.08489/) (2020)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [Generating Adversarial Examples with Adversarial Networks](https://mlsanity.com/paper/1801.02610/) (2018)
- [Compositional Explanations of Neurons](https://mlsanity.com/paper/2006.14032/) (2020)
- [AdvHat: Real-world adversarial attack on ArcFace Face ID system](https://mlsanity.com/paper/1908.08705/) (2019)
- [Generative Adversarial Perturbations](https://mlsanity.com/paper/1712.02328/) (2017)
- [Constructing Unrestricted Adversarial Examples with Generative Models](https://mlsanity.com/paper/1805.07894/) (2018)
- [Unrestricted Adversarial Examples via Semantic Manipulation](https://mlsanity.com/paper/1904.06347/) (2019)
- [Explanation by Progressive Exaggeration](https://mlsanity.com/paper/1911.00483/) (2019)
- [PhysGAN: Generating Physical-World-Resilient Adversarial Examples for Autonomous Driving](https://mlsanity.com/paper/1907.04449/) (2019)
- [Semantic Adversarial Attacks: Parametric Transformations That Fool Deep Classifiers](https://mlsanity.com/paper/1904.08489/) (2019)

## Built on by (most cited first, 27 of 27)

- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Physical Adversarial Attack meets Computer Vision: A Decade Survey](https://mlsanity.com/paper/2209.15179/) (2022)
- [Adversarial Training for High-Stakes Reliability](https://mlsanity.com/paper/2205.01663/) (2022)
- [Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning](https://mlsanity.com/paper/2208.08831/) (2022)
- [A Survey on Physical Adversarial Attack in Computer Vision](https://mlsanity.com/paper/2209.14262/) (2022)
- [Red Teaming Deep Neural Networks with Feature Synthesis Tools](https://mlsanity.com/paper/2302.10894/) (2023)
- [Physical Adversarial Attacks For Camera-based Smart Systems: Current Trends, Categorization, Applications, Research Challenges, and Future Outlook](https://mlsanity.com/paper/2308.06173/) (2023)
- [MERIT: Multi-view evidential learning for reliable and interpretable liver fibrosis staging](https://mlsanity.com/paper/2405.02918/) (2024)
- [DiG-IN: Diffusion Guidance for Investigating Networks -- Uncovering Classifier Differences Neuron Visualisations and Visual Counterfactual Explanations](https://mlsanity.com/paper/2311.17833/) (2023)
- [Adversarial Neon Beam: A Light-based Physical Attack to DNNs](https://mlsanity.com/paper/2204.00853/) (2022)
- [Diagnostics for Deep Neural Networks with Automated Copy/Paste Attacks](https://mlsanity.com/paper/2211.10024/) (2022)
- [Investigating the Adversarial Robustness of Density Estimation Using the Probability Flow ODE](https://mlsanity.com/paper/2310.07084/) (2023)
- [Transcending Adversarial Perturbations: Manifold-Aided Adversarial Examples with Legitimate Semantics](https://mlsanity.com/paper/2402.03095/) (2024)
- [Beyond Vulnerabilities: A Survey of Adversarial Attacks as Both Threats and Defenses in Computer Vision Systems](https://mlsanity.com/paper/2508.01845/) (2025)
- [How to Train your Antivirus: RL-based Hardening through the Problem-Space](https://mlsanity.com/paper/2402.19027/) (2024)
- [The SaTML '24 CNN Interpretability Competition: New Innovations for Concept-Level Interpretability](https://mlsanity.com/paper/2404.02949/) (2024)
- [Corrupting Neuron Explanations of Deep Visual Features](https://mlsanity.com/paper/2310.16332/) (2023)
- [Adversarial Catoptric Light: An Effective, Stealthy and Robust Physical-World Attack to DNNs](https://mlsanity.com/paper/2209.11739/) (2022)
- [MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents](https://mlsanity.com/paper/2412.08014/) (2024)
- [A Survey of Adversarial Defenses in Vision-based Systems: Categorization, Methods and Challenges](https://mlsanity.com/paper/2503.00384/) (2025)
- [SABER: Spatially Consistent 3D Universal Adversarial Objects for BEV Detectors](https://mlsanity.com/paper/2505.22499/) (2025)
- [Kaleidoscopic Background Attack: Disrupting Pose Estimation with Multi-Fold Radial Symmetry Textures](https://mlsanity.com/paper/2507.10265/) (2025)
- [Adversarial Doodles: Interpretable and Human-drawable Attacks Provide Describable Insights](https://mlsanity.com/paper/2311.15994/) (2023)
- [Exploring higher-order neural network node interactions with total correlation](https://mlsanity.com/paper/2402.04440/) (2024)
- [SafeFix: Targeted Model Repair via Controlled Image Generation](https://mlsanity.com/paper/2508.08701/) (2025)

## Abstract (arXiv)

The literature on adversarial attacks in computer vision typically focuses on pixel-level perturbations. These tend to be very difficult to interpret. Recent work that manipulates the latent representations of image generators to create "feature-level" adversarial perturbations gives us an opportunity to explore perceptible, interpretable adversarial attacks. We make three contributions. First, we observe that feature-level attacks provide useful classes of inputs for studying representations in models. Second, we show that these adversaries are uniquely versatile and highly robust. We demonstrate that they can be used to produce targeted, universal, disguised, physically-realizable, and black-box attacks at the ImageNet scale. Third, we show how these adversarial images can be used as a practical interpretability tool for identifying bugs in networks. We use these adversaries to make predictions about spurious associations between features and classes which we then test by designing "copy/paste" attacks in which one natural image is pasted into another to cause a targeted misclassification. Our results suggest that feature-level attacks are a promising approach for rigorous interpretability research. They support the design of tools to better understand what a model has learned and diagnose brittle feature associations. Code is available at https://github.com/thestephencasper/feature_level_adv