mlsanityData to 7 Oct 2026

What makes an image realistic?

arXiv 2403.04493 ↗First posted 7 Mar 2024cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

The last decade has seen tremendous progress in our ability to generate realistic-looking data, be it images, text, audio, or video. Here, we discuss the closely related problem of quantifying realism, that is, designing functions that can reliably tell realistic data from unrealistic data. This problem turns out to be significantly harder to solve and remains poorly understood, despite its prevalence in machine learning and recent breakthroughs in generative AI. Drawing on insights from algorithmic information theory, we discuss why this problem is challenging, why a good generative model alone is insufficient to solve it, and what a good solution would look like. In particular, we introduce the notion of a universal critic, which unlike adversarial critics does not require adversarial training. While universal critics are not immediately practical, they can serve both as a North Star for guiding practical implementations and as a tool for analyzing existing attempts to capture realism.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 32 of its 73 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

32 of 73 references.

References · 32

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  6. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  7. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  8. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  9. 2017The Perception-Distortion Tradeoffcited by 582
  10. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  11. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  12. 2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 423
20 more
  1. 2015A note on the evaluation of generative modelscited by 291
  2. 2015Generative Moment Matching Networkscited by 252
  3. 2022Diffusion models as plug-and-play priorscited by 230
  4. 2018Pros and Cons of GAN Evaluation Measurescited by 224
  5. 2015Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networkscited by 214
  6. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  7. 2019Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoffcited by 207
  8. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  9. 2015Training generative neural networks via Maximum Mean Discrepancy optimizationcited by 149
  10. 2020Nonlinear Transform Codingcited by 132
  11. 2023Towards image compression with perfect realism at ultra-low bitratescited by 99
  12. 2020Interacting particle solutions of Fokker-Planck equations through gradient-log-density estimationcited by 87
  13. 2022An Introduction to Neural Data Compressioncited by 78
  14. 2016Amortised MAP Inference for Image Super-resolutioncited by 68
  15. 2020Perfect density models cannot guarantee anomaly detectioncited by 39
  16. 2021KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Supportcited by 37
  17. 2021A coding theorem for the rate-distortion-perception functioncited by 30
  18. 2021On the advantages of stochastic encoderscited by 28
  19. 2022Out-of-Distribution Detection with Reconstruction Error and Typicality-based Penaltycited by 11
  20. 2023On the Choice of Perception Loss Function for Learned Video Compressioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 2.

10 papers.

  1. 2025Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenizationcited by 35
  2. 2025Generative Uncertainty in Diffusion Modelscited by 20
  3. 2025SAGI: Semantically Aligned and Uncertainty Guided AI Image Inpaintingcited by 10
  4. 2025Generative Latent Video Compressioncited by 8
  5. 2024Some Notes on the Sample Complexity of Approximate Channel Simulationcited by 4
  6. 2025You Cannot Feed Two Birds with One Score: the Accuracy-Naturalness Tradeoff in Translationcited by 4
  7. 2025Generative AI Meets 6G and Beyond: Diffusion Models for Semantic Communicationscited by 1
  8. 2025Keep It Real: Challenges in Attacking Compression-Based Adversarial Purificationcited by 0
  9. 2026Beyond Semantics: Uncovering the Physics of Fakes via Universal Physical Descriptors for Cross-Modal Synthetic Detectioncited by 0
  10. 2026CV-Arena: An Open Benchmark for Instructional Computer Vision Problem Solving with Human-AI Collaborative Preferencescited by 0