# Adversarial Robustness as a Prior for Learned Representations

arXiv: https://arxiv.org/abs/1906.00945 · HTML: https://mlsanity.com/paper/1906.00945/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 20 of 20)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [Enhance the Visual Representation via Discrete Adversarial Training](https://mlsanity.com/paper/2209.07735/) (2022)
- [Adversarial Training is Not Ready for Robot Learning](https://mlsanity.com/paper/2103.08187/) (2021)
- [Attack to Fool and Explain Deep Networks](https://mlsanity.com/paper/2106.10606/) (2021)
- [Instruct2Attack: Language-Guided Semantic Adversarial Attacks](https://mlsanity.com/paper/2311.15551/) (2023)
- [LOTUS: Evasive and Resilient Backdoor Attacks through Sub-Partitioning](https://mlsanity.com/paper/2403.17188/) (2024)
- [Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers](https://mlsanity.com/paper/2103.07470/) (2021)
- [Vec2Face-v2: Unveil Human Faces from their Blackbox Features via Attention-based Network in Face Recognition](https://mlsanity.com/paper/2209.04920/) (2022)
- [ImageNet Pre-training also Transfers Non-Robustness](https://mlsanity.com/paper/2106.10989/) (2021)
- [Addressing Mistake Severity in Neural Networks with Semantic Knowledge](https://mlsanity.com/paper/2211.11880/) (2022)
- [CROVIA: Seeing Drone Scenes from Car Perspective via Cross-View Adaptation](https://mlsanity.com/paper/2304.07199/) (2023)
- [Text-to-Image Generation Via Energy-Based CLIP](https://mlsanity.com/paper/2408.17046/) (2024)
- [Brain-like representational straightening of natural movies in robust feedforward neural networks](https://mlsanity.com/paper/2308.13870/) (2023)
- [Robustness and invariance properties of image classifiers](https://mlsanity.com/paper/2209.02408/) (2022)
- [Hard-label Manifolds: Unexpected Advantages of Query Efficiency for Finding On-manifold Adversarial Examples](https://mlsanity.com/paper/2103.03325/) (2021)
- [Toward Understanding Adversarial Distillation: Why Robust Teachers Fail](https://mlsanity.com/paper/2605.21999/) (2026)
- [Beyond False Stability: High-Noise Drift Gating for Test-Time Adversarial Defenses in Vision-Language Models](https://mlsanity.com/paper/2606.03730/) (2026)
- [Foveation-Guided Dynamic Token Selection for Robust and Efficient Vision Transformers](https://mlsanity.com/paper/2607.09480/) (2026)
- [Adaptive Model Inversion Attacks Generalize a Privacy-Robustness Tradeoff](https://mlsanity.com/paper/2610.07677/) (2026)

## Abstract (arXiv)

An important goal in deep learning is to learn versatile, high-level feature representations of input data. However, standard networks' representations seem to possess shortcomings that, as we illustrate, prevent them from fully realizing this goal. In this work, we show that robust optimization can be re-cast as a tool for enforcing priors on the features learned by deep neural networks. It turns out that representations learned by robust models address the aforementioned shortcomings and make significant progress towards learning a high-level encoding of inputs. In particular, these representations are approximately invertible, while allowing for direct visualization and manipulation of salient input features. More broadly, our results indicate adversarial robustness as a promising avenue for improving learned representations. Our code and models for reproducing these results is available at https://git.io/robust-reps .