mlsanityData to 7 Oct 2026

Pros and Cons of GAN Evaluation Measures

arXiv 1802.03446 ↗First posted 9 Feb 2018cs.CVcited by 224

Not on the map. Why?

Cited by 23 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generative models, in particular generative adversarial networks (GANs), have received significant attention recently. A number of GAN variants have been proposed and have been utilized in many applications. Despite large strides in terms of theoretical progress, evaluating and comparing GANs remains a daunting task. While several measures have been introduced, as of yet, there is no consensus as to which measure best captures strengths and limitations of models and should be used for fair model comparison. As in other areas of computer vision and machine learning, it is critical to settle on one or few good measures to steer the progress in this field. In this paper, I review and critically discuss more than 24 quantitative and 5 qualitative measures for evaluating generative models with a particular emphasis on GAN-derived models. I also provide a set of 7 desiderata followed by an evaluation of whether a given measure or a family of measures is compatible with them.

Key facts. Cited by 224 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 70; 2022: 63; 2023: 41; 2024: 24; 2025: 12; 2026: 14.

224 papers.

Sort:

By area:

  1. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  2. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 199
  3. 2021How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Modelscited by 193
  4. 2023A Categorical Archive of ChatGPT Failurescited by 167
  5. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  6. 2021Pros and Cons of GAN Evaluation Measures: New Developmentscited by 121
  7. 2021LOLNeRF: Learn from One Lookcited by 84
  8. 2022Diffusion Probabilistic Models beat GANs on Medical Imagescited by 77
  9. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codescited by 71
  10. 2021GANs for Medical Image Synthesis: An Empirical Studycited by 64
  11. 2022What Makes Transfer Learning Work For Medical Images: Feature Reuse & Other Factorscited by 61
  12. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorialcited by 60
38 more
  1. 2022Generated Faces in the Wild: Quantitative Comparison of Stable Diffusion, Midjourney and DALL-E 2cited by 55
  2. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
  3. 2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51
  4. 2021Generating Synthetic Mixed-type Longitudinal Electronic Health Records for Artificial Intelligent Applicationscited by 47
  5. 2022Data Augmentation techniques in time series domain: A survey and taxonomycited by 41
  6. 2022StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesiscited by 38
  7. 2021On the Limitations of Multimodal VAEscited by 37
  8. 2021Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systemscited by 36
  9. 2021Selective Synthetic Augmentation with HistoGAN for Improved Histopathology Image Classificationcited by 35
  10. 2021Manifold Topology Divergence: a Framework for Comparing Data Manifoldscited by 31
  11. 2022Studying Bias in GANs through the Lens of Racecited by 31
  12. 2023Qualitative Failures of Image Generation Models and Their Application in Detecting Deepfakescited by 31
  13. 2023Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Datacited by 31
  14. 2024Contractive Diffusion Probabilistic Modelscited by 29
  15. 2022InstaFormer: Instance-Aware Image-to-Image Translation with Transformercited by 28
  16. 2024Video Diffusion Models: A Surveycited by 28
  17. 2021Data Augmentation in High Dimensional Low Sample Size Setting Using a Geometry-Based Variational Autoencodercited by 27
  18. 2021BIPS: Bi-modal Indoor Panorama Synthesis via Residual Depth-aided Adversarial Learningcited by 26
  19. 2022A Survey on Training Challenges in Generative Adversarial Networks for Biomedical Image Analysiscited by 24
  20. 2023Feature Extraction for Generative Medical Imaging Evaluation: New Evidence Against an Evolving Trendcited by 24
  21. 2023Beware of diffusion models for synthesizing medical images -- A comparison with GANs in terms of memorizing brain MRI and chest x-ray imagescited by 23
  22. 2021Regularising Inverse Problems with Generative Machine Learning Modelscited by 22
  23. 2023Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Designcited by 22
  24. 2021Generative adversarial networks in time series: A survey and taxonomycited by 20
  25. 2021Incentivizing Collaboration in Machine Learning via Synthetic Data Rewardscited by 20
  26. 2022Why is constrained neural language generation particularly challenging?cited by 19
  27. 2023Brain tumor segmentation using synthetic MR images -- A comparison of GANs and diffusion modelscited by 19
  28. 2021On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competitioncited by 18
  29. 2021A deep generative model for probabilistic energy forecasting in power systems: normalizing flowscited by 18
  30. 2023TIAM -- A Metric for Evaluating Alignment in Text-to-Image Generationcited by 18
  31. 2021Data synthesis and adversarial networks: A review and meta-analysis in cancer imagingcited by 15
  32. 2022A survey on GANs for computer vision: Recent research, analysis and taxonomycited by 15
  33. 2021Generative Modeling for Multi-task Visual Learningcited by 14
  34. 2022medigan: a Python library of pretrained generative models for medical image synthesiscited by 14
  35. 2023Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustnesscited by 14
  36. 2023Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-artcited by 14
  37. 2024A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Sciencecited by 14
  38. 2021Generating Physically-Consistent Satellite Imagery for Climate Visualizationscited by 13

50 most cited of 224 All 224 (JSON)