mlsanityData to 14 Oct 2026

A Pragmatic Note on Evaluating Generative Models with Fr'echet Inception Distance for Retinal Image Synthesis

arXiv 2502.17160 ↗First posted 24 Feb 2025cs.CV cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Fr'echet Inception Distance (FID), computed with an ImageNet pretrained Inception-v3 network, is widely used as a state-of-the-art evaluation metric for generative models. It assumes that feature vectors from Inception-v3 follow a multivariate Gaussian distribution and calculates the 2-Wasserstein distance based on their means and covariances. While FID effectively measures how closely synthetic data match real data in many image synthesis tasks, the primary goal in biomedical generative models is often to enrich training datasets ideally with corresponding annotations. For this purpose, the gold standard for evaluating generative models is to incorporate synthetic data into downstream task training, such as classification and segmentation, to pragmatically assess its performance. In this paper, we examine cases from retinal imaging modalities, including color fundus photography and optical coherence tomography, where FID and its related metrics misalign with task-specific evaluation goals in classification and segmentation. We highlight the limitations of using various metrics, represented by FID and its variants, as evaluation criteria for these applications and address their potential caveats in broader biomedical imaging modalities and downstream tasks.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 34 of its 53 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

34 of 53 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  9. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  10. 2017Neural Discrete Representation Learningcited by 4,712
  11. 2016Improved Techniques for Training GANscited by 2,970
  12. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
22 more
  1. 2018Demystifying MMD GANscited by 1,293
  2. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  3. 2020U2-Net: Going Deeper with Nested U-Structure for Salient Object Detectioncited by 412
  4. 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 310
  5. 2015A note on the evaluation of generative modelscited by 292
  6. 2019Effectively Unbiased FID and Inception Score and where to find themcited by 165
  7. 2021Explaining in Style: Training a GAN to explain a classifier in StyleSpacecited by 122
  8. 2022Diffusion Probabilistic Models beat GANs on Medical Imagescited by 77
  9. 2022GAMMA Challenge:Glaucoma grAding from Multi-Modality imAgescited by 46
  10. 2023AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challengecited by 35
  11. 2018An Improved Evaluation Framework for Generative Adversarial Networkscited by 26
  12. 2023Feature Likelihood Divergence: Evaluating the Generalization of Generative Models Using Samplescited by 25
  13. 2023Feature Extraction for Generative Medical Imaging Evaluation: New Evidence Against an Evolving Trendcited by 24
  14. 2024Fr'echet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasetscited by 19
  15. 2022Dataset and Evaluation algorithm design for GOALS Challengecited by 13
  16. 2023Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Setscited by 13
  17. 2021Evaluating generative networks using Gaussian mixtures of image featurescited by 11
  18. 2024Disentangling representations of retinal images with generative modelscited by 8
  19. 2024Annotated Biomedical Video Generation using Denoising Diffusion Probabilistic Models and Flow Fieldscited by 5
  20. 2023Retinal OCT Synthesis with Denoising Diffusion Probabilistic Models for Layer Segmentationcited by 4
  21. 2024Rethinking Perceptual Metrics for Medical Image Translationcited by 4
  22. 2024Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

  1. 2025MRI-CORE: A Foundation Model for Magnetic Resonance Imagingcited by 15
  2. 2025Quantifying the uncertainty of model-based synthetic image quality metricscited by 2
  3. 2026The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluationcited by 2
  4. 2025CytoDiff: AI-Driven Cytomorphology Image Synthesis for Medical Diagnosticscited by 1
  5. 2026SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identificationcited by 1
  6. 2026Evaluating the trustworthiness of the Fr'echet Inception Distance with stochastic embedding representationscited by 0
  7. 2026Ordinal Diffusion Models for Color Fundus Imagescited by 0
  8. 2026Trustworthy deep domain adaptation for wearable photoplethysmography signal analysis with decision-theoretic uncertainty quantificationcited by 0
  9. 2026Diffusion-Based Tumor Inpainting for Renal Segmentation under Clinical Data Scarcitycited by 0