mlsanityData to 14 Oct 2026

A note on the evaluation of generative models

arXiv 1511.01844 ↗First posted 5 Nov 2015stat.ML cs.LGcited by 292

Not on the map. Why?

Cited by 57 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Probabilistic generative models can be used for compression, denoising, inpainting, texture synthesis, semi-supervised learning, unsupervised feature learning, and other tasks. Given this wide range of applications, it is not surprising that a lot of heterogeneity exists in the way these models are formulated, trained, and evaluated. As a consequence, direct comparison between models is often difficult. This article reviews mostly known but often underappreciated properties relating to the evaluation and interpretation of generative models with a focus on image models. In particular, we show that three of the currently most commonly used criteria---average log-likelihood, Parzen window estimates, and visual fidelity of samples---are largely independent of each other when the data is high-dimensional. Good performance with respect to one criterion therefore need not imply good performance with respect to the other criteria. Our results show that extrapolation from one criterion to another is not warranted and generative models need to be evaluated directly with respect to the application(s) they were intended for. In addition, we provide examples demonstrating that Parzen window estimates should generally be avoided.

Key facts. Cited by 292 papers in the map (data through 14 Oct 2026). First posted on arXiv 5 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 57; 2022: 68; 2023: 56; 2024: 39; 2025: 34; 2026: 38.

292 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 672
  3. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  4. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 373
  5. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 337
  6. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Modelscited by 334
  7. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
  8. 2022A Survey on Generative Diffusion Modelcited by 240
  9. 2021How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Modelscited by 196
  10. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 190
  11. 2021Challenges in Detoxifying Language Modelscited by 169
  12. 2021Pros and Cons of GAN Evaluation Measures: New Developmentscited by 123
38 more
  1. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 120
  2. 2024Why Tabular Foundation Models Should Be a Research Prioritycited by 106
  3. 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 102
  4. 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 100
  5. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimationcited by 97
  6. 2023KoLA: Carefully Benchmarking World Knowledge of Large Language Modelscited by 85
  7. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Modelscited by 83
  8. 2022An Introduction to Neural Data Compressioncited by 79
  9. 2025OmniSVG: A Unified Scalable Vector Graphics Generation Modelcited by 74
  10. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codescited by 71
  11. 2021Predicting Video with VQVAEcited by 61
  12. 2021On Memorization in Probabilistic Deep Generative Modelscited by 60
  13. 2021Hierarchical VAEs Know What They Don't Knowcited by 58
  14. 2021Diffusion-Based Representation Learningcited by 57
  15. 2021Score-Based Generative Classifierscited by 53
  16. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
  17. 2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51
  18. 2023Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learningcited by 48
  19. 2024Categorical Flow Matching on Statistical Manifoldscited by 47
  20. 2022On Evaluation Metrics for Graph Generative Modelscited by 46
  21. 2024JetFormer: An Autoregressive Generative Model of Raw Images and Textcited by 46
  22. 2021Class-Incremental Learning with Generative Classifierscited by 39
  23. 2021Copula Flows for Synthetic Data Generationcited by 35
  24. 2021Protein sequence design with deep generative modelscited by 35
  25. 2022How Much is Enough? A Study on Diffusion Times in Score-based Generative Modelscited by 35
  26. 2023A Survey on Graph Diffusion Models: Generative AI in Science for Molecule, Protein and Materialcited by 35
  27. 2022When do GANs replicate? On the choice of dataset sizecited by 34
  28. 2021Manifold Topology Divergence: a Framework for Comparing Data Manifoldscited by 31
  29. 2024On the Challenges and Opportunities in Generative AIcited by 31
  30. 2023Synthetic data, real errors: how (not) to publish and use synthetic datacited by 29
  31. 2023Bayesian Conditional Diffusion Models for Versatile Spatiotemporal Turbulence Generationcited by 29
  32. 2022Modeling sRGB Camera Noise with Normalizing Flowscited by 28
  33. 2023Feature Likelihood Divergence: Evaluating the Generalization of Generative Models Using Samplescited by 25
  34. 2021Conditional Invertible Neural Networks for Diverse Image-to-Image Translationcited by 24
  35. 2025Generative Uncertainty in Diffusion Modelscited by 23
  36. 2021Regularising Inverse Problems with Generative Machine Learning Modelscited by 22
  37. 2021Unconditional Scene Graph Generationcited by 22
  38. 2021A Structural Causal Model for MR Images of Multiple Sclerosiscited by 21

50 most cited of 292 All 292 (JSON)