mlsanityData to 14 Oct 2026

Intriguing properties of generative classifiers

arXiv 2309.16779 ↗First posted 28 Sep 2023cs.CV cs.AI cs.LGcited by 52

Not on the map. Why?

Cited by 21 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

What is the best paradigm to recognize objects -- discriminative inference (fast but potentially prone to shortcut learning) or using a generative model (slow but potentially more robust)? We build on recent advances in generative modeling that turn text-to-image models into classifiers. This allows us to study their behavior and to compare them against discriminative models and human psychophysical data. We report four intriguing emergent properties of generative classifiers: they show a record-breaking human-like shape bias (99% for Imagen), near human-level out-of-distribution accuracy, state-of-the-art alignment with human classification errors, and they understand certain perceptual illusions. Our results indicate that while the current dominant paradigm for modeling human object recognition is discriminative inference, zero-shot generative models approximate human object recognition data surprisingly well.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

30 of 47 references.

References · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2013Auto-Encoding Variational Bayescited by 4,053
  10. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  11. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  12. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
18 more
  1. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  2. 2021Variational Diffusion Modelscited by 1,153
  3. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  4. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  5. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  6. 2018Recognition in Terra Incognitacited by 658
  7. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  8. 2022Understanding Diffusion Models: A Unified Perspectivecited by 340
  9. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 269
  10. 2021Partial success in closing the gap between human and machine visioncited by 185
  11. 2023Text-to-Image Diffusion Models are Zero-Shot Classifierscited by 143
  12. 2018Towards the first adversarially robust neural network model on MNISTcited by 104
  13. 2023Image Captioners Are Scalable Vision Learners Toocited by 89
  14. 2020Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistencycited by 78
  15. 2021Score-Based Generative Classifierscited by 53
  16. 2023DreamTeacher: Pretraining Image Backbones with Deep Generative Modelscited by 37
  17. 2022Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentorscited by 34
  18. 2023Spatial-frequency channels, shape bias, and adversarial robustnesscited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 17; 2025: 19; 2026: 12.

52 papers.

Sort:

By area:

  1. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 279
  2. 2025Video models are zero-shot learners and reasonerscited by 220
  3. 2023Getting aligned on representational alignmentcited by 143
  4. 2025Unified Multimodal Discrete Diffusioncited by 49
  5. 2023Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Modelscited by 46
  6. 2024Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studiescited by 38
  7. 2024Artificial Kuramoto Oscillatory Neuronscited by 38
  8. 2023RadEdit: stress-testing biomedical vision models via diffusion image editingcited by 28
  9. 2024Can We Talk Models Into Seeing the World Differently?cited by 23
  10. 2024Factorized Diffusion: Perceptual Illusions by Noise Decompositioncited by 23
  11. 2024Can Biases in ImageNet Models Explain Generalization?cited by 20
  12. 2025Diffusion Counterfactual Generation with Semantic Abductioncited by 10
38 more
  1. 2024Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakerscited by 8
  2. 2025Generative Classifiers Avoid Shortcut Solutionscited by 8
  3. 2025Unleashing the Potential of Pre-Trained Diffusion Models for Generalizable Person Re-Identificationcited by 7
  4. 2025Diffusion Classifiers Understand Compositionality, but Conditions Applycited by 7
  5. 2024Diffusion-based Visual Anagram as Multi-task Learningcited by 6
  6. 2024An Analysis of Human Alignment of Latent Diffusion Modelscited by 5
  7. 2024Generative Edge Detection with Stable Diffusioncited by 5
  8. 2024Illusion3D: 3D Multiview Illusion with 2D Diffusion Priorscited by 5
  9. 2025Perceptual Inductive Bias Is What You Need Before Contrastive Learningcited by 4
  10. 2024Investigating the Effectiveness of Cross-Attention to Unlock Zero-Shot Editing of Text-to-Video Diffusion Modelscited by 3
  11. 2024Greedy Growing Enables High-Resolution Pixel-Based Diffusion Modelscited by 3
  12. 2024How Do Training Methods Influence the Utilization of Vision Models?cited by 3
  13. 2025Shape Bias and Robustness Evaluation via Cue Decomposition for Image Classification and Segmentationcited by 3
  14. 2025Compositional Scene Understanding through Inverse Generative Modelingcited by 3
  15. 2025Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Modelscited by 3
  16. 2024Towards Generative Class Prompt Learning for Fine-grained Visual Recognitioncited by 2
  17. 2025Exploring Structured Semantic Priors Underlying Diffusion Score for Test-time Adaptationcited by 2
  18. 2025Your VAR Model is Secretly an Efficient and Explainable Generative Classifiercited by 2
  19. 2023Layerwise complexity-matched learning yields an improved model of cortical area V2cited by 1
  20. 2024Detecting Out-of-Distribution Objects through Class-Conditioned Inpaintingcited by 1
  21. 2024Classification-Denoising Networkscited by 1
  22. 2025Latent Diffusion U-Net Representations Contain Positional Embeddings and Anomaliescited by 1
  23. 2025StarFT: Robust Fine-tuning of Zero-shot Models via Spuriosity Alignmentcited by 1
  24. 2025Noise-to-Notes: Diffusion-based Generation and Refinement for Automatic Drum Transcriptioncited by 1
  25. 2025Leveraging Prior Knowledge of Diffusion Model for Person Searchcited by 1
  26. 2025The Hidden Cost of Modeling P(X): Vulnerability to Membership Inference Attacks in Generative Text Classifierscited by 1
  27. 2025Is Generation Required for Data-Efficient Perception?cited by 1
  28. 2026One Model, Many Behaviors: Training-Induced Effects on Out-of-Distribution Detectioncited by 1
  29. 2026Low-Pass Filtering Improves Behavioral Alignment of Vision Modelscited by 1
  30. 2026On Discriminative vs. Generative classifiers: Rethinking MLLMs for Action Understandingcited by 1
  31. 2025Beyond Objects: Contextual Synthetic Data Generation for Fine-Grained Classificationcited by 0
  32. 2026Style-Aware Gloss Control for Generative Non-Photorealistic Renderingcited by 0
  33. 2026Expanding the Role of Diffusion Models for Robust Classifier Trainingcited by 0
  34. 2026Revisiting Autoregressive Models for Generative Image Classificationcited by 0
  35. 2026Emotion Diffusion Classifier with Adaptive Margin Discrepancy Training for Facial Expression Recognitioncited by 0
  36. 2026Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spotcited by 0
  37. 2026Quality-Preserving Imperceptible Adversarial Attack on Skeleton-based Human Action Recognitioncited by 0
  38. 2026Concept Removal Guidance: Evidence-Calibrated Negative Guidance for Safe Diffusion Samplingcited by 0

50 most cited of 52 All 52 (JSON)