# Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers

arXiv: https://arxiv.org/abs/2103.07470 · HTML: https://mlsanity.com/paper/2103.07470/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 15 Mar 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [Do Adversarially Robust ImageNet Models Transfer Better?](https://mlsanity.com/paper/2007.08489/) (2020)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [cGANs with Projection Discriminator](https://mlsanity.com/paper/1802.05637/) (2018)
- [Generating Images with Perceptual Similarity Metrics based on Deep Networks](https://mlsanity.com/paper/1602.02644/) (2016)
- [Inverting Visual Representations with Convolutional Networks](https://mlsanity.com/paper/1506.02753/) (2015)
- [Modulating early visual processing by language](https://mlsanity.com/paper/1707.00683/) (2017)
- [Adversarial Robustness through Local Linearization](https://mlsanity.com/paper/1907.02610/) (2019)
- [Boundless: Generative Adversarial Networks for Image Extension](https://mlsanity.com/paper/1908.07007/) (2019)
- [Semantic Pyramid for Image Generation](https://mlsanity.com/paper/2003.06221/) (2020)
- [Adversarial Robustness as a Prior for Learned Representations](https://mlsanity.com/paper/1906.00945/) (2019)
- [Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification](https://mlsanity.com/paper/1606.06582/) (2016)
- [Inverting Supervised Representations with Autoregressive Neural Density Models](https://mlsanity.com/paper/1806.00400/) (2018)
- [Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs](https://mlsanity.com/paper/2008.01777/) (2020)

## Built on by (most cited first, 11 of 11)

- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Text Embeddings Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2310.06816/) (2023)
- [Can Neural Network Memorization Be Localized?](https://mlsanity.com/paper/2307.09542/) (2023)
- [Language Model Inversion](https://mlsanity.com/paper/2311.13647/) (2023)
- [Spectral Bias in Practice: The Role of Function Frequency in Generalization](https://mlsanity.com/paper/2110.02424/) (2021)
- [Transferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queries](https://mlsanity.com/paper/2406.10280/) (2024)
- [Better Language Model Inversion by Compactly Representing Next-Token Distributions](https://mlsanity.com/paper/2506.17090/) (2025)
- [Privacy Vulnerability of Split Computing to Data-Free Model Inversion Attacks](https://mlsanity.com/paper/2107.06304/) (2021)
- [Mitigating Paraphrase Attacks on Machine-Text Detectors via Paraphrase Inversion](https://mlsanity.com/paper/2410.21637/) (2024)
- [Feature Space Analysis by Guided Diffusion Model](https://mlsanity.com/paper/2509.07936/) (2025)
- [The Layer Mystery of VLA: An Information-Theoretical Analysis of VLA Latent Interface](https://mlsanity.com/paper/2609.36118/) (2026)

## Abstract (arXiv)

A discriminatively trained neural net classifier can fit the training data perfectly if all information about its input other than class membership has been discarded prior to the output layer. Surprisingly, past research has discovered that some extraneous visual detail remains in the logit vector. This finding is based on inversion techniques that map deep embeddings back to images. We explore this phenomenon further using a novel synthesis of methods, yielding a feedforward inversion model that produces remarkably high fidelity reconstructions, qualitatively superior to those of past efforts. When applied to an adversarially robust classifier model, the reconstructions contain sufficient local detail and global structure that they might be confused with the original image in a quick glance, and the object category can clearly be gleaned from the reconstruction. Our approach is based on BigGAN (Brock, 2019), with conditioning on logits instead of one-hot class labels. We use our reconstruction model as a tool for exploring the nature of representations, including: the influence of model architecture and training objectives (specifically robust losses), the forms of invariance that networks achieve, representational differences between correctly and incorrectly classified images, and the effects of manipulating logits and images. We believe that our method can inspire future investigations into the nature of information flow in a neural net and can provide diagnostics for improving discriminative models.