mlsanityData to 7 Oct 2026

Effectively Unbiased FID and Inception Score and where to find them

arXiv 1911.07023 ↗First posted 16 Nov 2019cs.CV cs.LGcited by 161

Not on the map. Why?

Cited by 40 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper shows that two commonly used evaluation metrics for generative models, the Fr'echet Inception Distance (FID) and the Inception Score (IS), are biased -- the expected value of the score computed for a finite sample set is not the true value of the score. Worse, the paper shows that the bias term depends on the particular model being evaluated, so model A may get a better score than model B simply because model A's bias term is smaller. This effect cannot be fixed by evaluating at a fixed number of samples. This means all comparisons using FID or IS as currently computed are unreliable. We then show how to extrapolate the score to obtain an effectively bias-free estimate of scores computed with an infinite number of samples, which we term overlinetextrmFID_∞ and overlinetextrmIS_∞. In turn, this effectively bias-free estimate requires good estimates of scores with a finite number of samples. We show that using Quasi-Monte Carlo integration notably improves estimates of FID and IS for finite sample sets. Our extrapolated scores are simple, drop-in replacements for the finite sample scores. Additionally, we show that using low discrepancy sequence in GAN training offers small improvements in the resulting generator.

Key facts. Cited by 161 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 26; 2023: 28; 2024: 41; 2025: 20; 2026: 29.

161 papers.

Sort:

By area:

  1. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  2. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  3. 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 303
  4. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  5. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  6. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 199
  7. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  8. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  9. 2021Pros and Cons of GAN Evaluation Measures: New Developmentscited by 121
  10. 2024Why Tabular Foundation Models Should Be a Research Prioritycited by 103
  11. 2022Any-resolution Training for High-resolution Image Synthesiscited by 72
  12. 2024DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptionscited by 60
38 more
  1. 2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51
  2. 2021Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualizationcited by 51
  3. 2021JoJoGAN: One Shot Face Stylizationcited by 49
  4. 2022ArtFID: Quantitative Evaluation of Neural Style Transfercited by 49
  5. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  6. 2022A Study on the Evaluation of Generative Modelscited by 35
  7. 2024A survey of synthetic data augmentation methods in computer visioncited by 34
  8. 2021Toward Accurate and Realistic Outfits Visualization with Attention to Detailscited by 33
  9. 2022Guided Co-Modulated GAN for 360deg Field of View Extrapolationcited by 32
  10. 2022Evaluating generative models in high energy physicscited by 31
  11. 2024Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning?cited by 31
  12. 2024Getting it Right: Improving Spatial Consistency in Text-to-Image Modelscited by 30
  13. 2024Visual Fourier Prompt Tuningcited by 30
  14. 2022FlexIT: Towards Flexible Semantic Image Translationcited by 28
  15. 2022Pythae: Unifying Generative Autoencoders in Python -- A Benchmarking Use Casecited by 26
  16. 2025KAD: No More FAD! An Effective and Efficient Evaluation Metric for Audio Generationcited by 24
  17. 2024Beyond FVD: Enhanced Evaluation Metrics for Video Generation Qualitycited by 21
  18. 2023HWD: A Novel Evaluation Score for Styled Handwritten Text Generationcited by 19
  19. 2023NetDiffus: Network Traffic Generation by Diffusion Models through Time-Series Imagingcited by 18
  20. 2023Deep Generative Models for Detector Signature Simulation: A Taxonomic Reviewcited by 18
  21. 2024Diffusion Model Guided Sampling with Pixel-Wise Aleatoric Uncertainty Estimationcited by 17
  22. 2021Conditional Frechet Inception Distancecited by 16
  23. 2023Efficient Quantization Strategies for Latent Diffusion Modelscited by 16
  24. 2021Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attentioncited by 15
  25. 2021Retrieve in Style: Unsupervised Facial Feature Transfer and Retrievalcited by 15
  26. 2021Data synthesis and adversarial networks: A review and meta-analysis in cancer imagingcited by 15
  27. 2021StyleGAN of All Trades: Image Manipulation with Only Pretrained StyleGANcited by 14
  28. 2024A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Sciencecited by 14
  29. 2024Deep Generative Models for 3D Medical Image Synthesiscited by 14
  30. 2021GANs N' Roses: Stable, Controllable, Diverse Image to Image Translation (works for videos too!)cited by 13
  31. 2024The Surprising Effectiveness of Skip-Tuning in Diffusion Samplingcited by 13
  32. 2022DALLE-URBAN: Capturing the urban design expertise of large text to image transformerscited by 12
  33. 2025Metrics that matter: Evaluating image quality metrics for medical image generationcited by 12
  34. 2021Evaluating generative networks using Gaussian mixtures of image featurescited by 11
  35. 2022On the Robustness of Quality Measures for GANscited by 11
  36. 2025ForgetMe: Evaluating Selective Forgetting in Generative Modelscited by 11
  37. 2022Exploring Vision Transformers as Diffusion Learnerscited by 10
  38. 2025CAD-VAE: Leveraging Correlation-Aware Latents for Comprehensive Fair Disentanglementcited by 10

50 most cited of 161 All 161 (JSON)