# Exploring the Connection between Robust and Generative Models

arXiv: https://arxiv.org/abs/2304.04033 · HTML: https://mlsanity.com/paper/2304.04033/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 8 Apr 2023.

## Builds on (most cited first, 25 of 25)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality](https://mlsanity.com/paper/1801.02613/) (2018)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Detecting Adversarial Samples from Artifacts](https://mlsanity.com/paper/1703.00410/) (2017)
- [Torchattacks: A PyTorch Repository for Adversarial Attacks](https://mlsanity.com/paper/2010.01950/) (2020)
- [Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text](https://mlsanity.com/paper/2012.04698/) (2020)
- [Adversarial Example Detection for DNN Models: A Review and Experimental Comparison](https://mlsanity.com/paper/2105.00203/) (2021)
- [Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?](https://mlsanity.com/paper/1910.08640/) (2019)
- [JEM++: Improved Techniques for Training JEM](https://mlsanity.com/paper/2109.09032/) (2021)
- [Towards Understanding the Generative Capability of Adversarially Robust Classifiers](https://mlsanity.com/paper/2108.09093/) (2021)
- [Bridging Adversarial Robustness and Gradient Interpretability](https://mlsanity.com/paper/1903.11626/) (2019)
- [On the Benefits of Models with Perceptually-Aligned Gradients](https://mlsanity.com/paper/2005.01499/) (2020)
- [A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training](https://mlsanity.com/paper/2203.13455/) (2022)
- [Adversarial Training Reduces Information and Improves Transferability](https://mlsanity.com/paper/2007.11259/) (2020)
- [Inverting Adversarially Robust Networks for Image Synthesis](https://mlsanity.com/paper/2106.06927/) (2021)
- [MAGIC: Mask-Guided Image Synthesis by Inverting a Quasi-Robust Classifier](https://mlsanity.com/paper/2209.11549/) (2022)
- [Adversarial Training Improves Joint Energy-Based Generative Modelling](https://mlsanity.com/paper/2207.08950/) (2022)

## Built on by (most cited first, 2 of 2)

- [Shedding More Light on Robust Classifiers under the lens of Energy-based Models](https://mlsanity.com/paper/2407.06315/) (2024)
- [Understanding Adversarial Training with Energy-based Models](https://mlsanity.com/paper/2505.22486/) (2025)

## Abstract (arXiv)

We offer a study that connects robust discriminative classifiers trained with adversarial training (AT) with generative modeling in the form of Energy-based Models (EBM). We do so by decomposing the loss of a discriminative classifier and showing that the discriminative model is also aware of the input data density. Though a common assumption is that adversarial points leave the manifold of the input data, our study finds out that, surprisingly, untargeted adversarial points in the input space are very likely under the generative model hidden inside the discriminative classifier -- have low energy in the EBM. We present two evidence: untargeted attacks are even more likely than the natural data and their likelihood increases as the attack strength increases. This allows us to easily detect them and craft a novel attack called High-Energy PGD that fools the classifier yet has energy similar to the data set. The code is available at github.com/senad96/Robust-Generative