# Intriguing properties of generative classifiers

arXiv: https://arxiv.org/abs/2309.16779 · HTML: https://mlsanity.com/paper/2309.16779/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 28 Sep 2023.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [Unleashing Text-to-Image Diffusion Models for Visual Perception](https://mlsanity.com/paper/2303.02153/) (2023)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Text-to-Image Diffusion Models are Zero-Shot Classifiers](https://mlsanity.com/paper/2303.15233/) (2023)
- [Towards the first adversarially robust neural network model on MNIST](https://mlsanity.com/paper/1805.09190/) (2018)
- [Image Captioners Are Scalable Vision Learners Too](https://mlsanity.com/paper/2306.07915/) (2023)
- [Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency](https://mlsanity.com/paper/2006.16736/) (2020)
- [Score-Based Generative Classifiers](https://mlsanity.com/paper/2110.00473/) (2021)
- [DreamTeacher: Pretraining Image Backbones with Deep Generative Models](https://mlsanity.com/paper/2307.07487/) (2023)
- [Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentors](https://mlsanity.com/paper/2211.13224/) (2022)
- [Spatial-frequency channels, shape bias, and adversarial robustness](https://mlsanity.com/paper/2309.13190/) (2023)

## Built on by (most cited first, 30 of 52)

- [Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation](https://mlsanity.com/paper/2403.12015/) (2024)
- [Video models are zero-shot learners and reasoners](https://mlsanity.com/paper/2509.20328/) (2025)
- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Unified Multimodal Discrete Diffusion](https://mlsanity.com/paper/2503.20853/) (2025)
- [Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Models](https://mlsanity.com/paper/2311.17919/) (2023)
- [Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies](https://mlsanity.com/paper/2404.09349/) (2024)
- [Artificial Kuramoto Oscillatory Neurons](https://mlsanity.com/paper/2410.13821/) (2024)
- [RadEdit: stress-testing biomedical vision models via diffusion image editing](https://mlsanity.com/paper/2312.12865/) (2023)
- [Can We Talk Models Into Seeing the World Differently?](https://mlsanity.com/paper/2403.09193/) (2024)
- [Factorized Diffusion: Perceptual Illusions by Noise Decomposition](https://mlsanity.com/paper/2404.11615/) (2024)
- [Can Biases in ImageNet Models Explain Generalization?](https://mlsanity.com/paper/2404.01509/) (2024)
- [Diffusion Counterfactual Generation with Semantic Abduction](https://mlsanity.com/paper/2506.07883/) (2025)
- [Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakers](https://mlsanity.com/paper/2403.07214/) (2024)
- [Generative Classifiers Avoid Shortcut Solutions](https://mlsanity.com/paper/2512.25034/) (2025)
- [Unleashing the Potential of Pre-Trained Diffusion Models for Generalizable Person Re-Identification](https://mlsanity.com/paper/2502.06619/) (2025)
- [Diffusion Classifiers Understand Compositionality, but Conditions Apply](https://mlsanity.com/paper/2505.17955/) (2025)
- [Diffusion-based Visual Anagram as Multi-task Learning](https://mlsanity.com/paper/2412.02693/) (2024)
- [An Analysis of Human Alignment of Latent Diffusion Models](https://mlsanity.com/paper/2403.08469/) (2024)
- [Generative Edge Detection with Stable Diffusion](https://mlsanity.com/paper/2410.03080/) (2024)
- [Illusion3D: 3D Multiview Illusion with 2D Diffusion Priors](https://mlsanity.com/paper/2412.09625/) (2024)
- [Perceptual Inductive Bias Is What You Need Before Contrastive Learning](https://mlsanity.com/paper/2506.01201/) (2025)
- [Investigating the Effectiveness of Cross-Attention to Unlock Zero-Shot Editing of Text-to-Video Diffusion Models](https://mlsanity.com/paper/2404.05519/) (2024)
- [Greedy Growing Enables High-Resolution Pixel-Based Diffusion Models](https://mlsanity.com/paper/2405.16759/) (2024)
- [How Do Training Methods Influence the Utilization of Vision Models?](https://mlsanity.com/paper/2410.14470/) (2024)
- [Shape Bias and Robustness Evaluation via Cue Decomposition for Image Classification and Segmentation](https://mlsanity.com/paper/2503.12453/) (2025)
- [Compositional Scene Understanding through Inverse Generative Modeling](https://mlsanity.com/paper/2505.21780/) (2025)
- [Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models](https://mlsanity.com/paper/2507.00493/) (2025)
- [Towards Generative Class Prompt Learning for Fine-grained Visual Recognition](https://mlsanity.com/paper/2409.01835/) (2024)
- [Exploring Structured Semantic Priors Underlying Diffusion Score for Test-time Adaptation](https://mlsanity.com/paper/2501.00873/) (2025)
- [Your VAR Model is Secretly an Efficient and Explainable Generative Classifier](https://mlsanity.com/paper/2510.12060/) (2025)

## Abstract (arXiv)

What is the best paradigm to recognize objects -- discriminative inference (fast but potentially prone to shortcut learning) or using a generative model (slow but potentially more robust)? We build on recent advances in generative modeling that turn text-to-image models into classifiers. This allows us to study their behavior and to compare them against discriminative models and human psychophysical data. We report four intriguing emergent properties of generative classifiers: they show a record-breaking human-like shape bias (99% for Imagen), near human-level out-of-distribution accuracy, state-of-the-art alignment with human classification errors, and they understand certain perceptual illusions. Our results indicate that while the current dominant paradigm for modeling human object recognition is discriminative inference, zero-shot generative models approximate human object recognition data surprisingly well.