# A Pragmatic Note on Evaluating Generative Models with Fr'echet Inception Distance for Retinal Image Synthesis

arXiv: https://arxiv.org/abs/2502.17160 · HTML: https://mlsanity.com/paper/2502.17160/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 53 references that are in the map.
- First posted on arXiv 24 Feb 2025.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [Effectively Unbiased FID and Inception Score and where to find them](https://mlsanity.com/paper/1911.07023/) (2019)
- [Explaining in Style: Training a GAN to explain a classifier in StyleSpace](https://mlsanity.com/paper/2104.13369/) (2021)
- [Diffusion Probabilistic Models beat GANs on Medical Images](https://mlsanity.com/paper/2212.07501/) (2022)
- [GAMMA Challenge:Glaucoma grAding from Multi-Modality imAges](https://mlsanity.com/paper/2202.06511/) (2022)
- [AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge](https://mlsanity.com/paper/2302.01738/) (2023)
- [An Improved Evaluation Framework for Generative Adversarial Networks](https://mlsanity.com/paper/1803.07474/) (2018)
- [Feature Likelihood Divergence: Evaluating the Generalization of Generative Models Using Samples](https://mlsanity.com/paper/2302.04440/) (2023)
- [Feature Extraction for Generative Medical Imaging Evaluation: New Evidence Against an Evolving Trend](https://mlsanity.com/paper/2311.13717/) (2023)
- [Fr'echet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasets](https://mlsanity.com/paper/2412.01496/) (2024)
- [Dataset and Evaluation algorithm design for GOALS Challenge](https://mlsanity.com/paper/2207.14447/) (2022)
- [Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets](https://mlsanity.com/paper/2301.10227/) (2023)
- [Evaluating generative networks using Gaussian mixtures of image features](https://mlsanity.com/paper/2110.05240/) (2021)
- [Disentangling representations of retinal images with generative models](https://mlsanity.com/paper/2402.19186/) (2024)

## Built on by (most cited first, 9 of 9)

- [MRI-CORE: A Foundation Model for Magnetic Resonance Imaging](https://mlsanity.com/paper/2506.12186/) (2025)
- [Quantifying the uncertainty of model-based synthetic image quality metrics](https://mlsanity.com/paper/2504.03623/) (2025)
- [The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation](https://mlsanity.com/paper/2606.20536/) (2026)
- [CytoDiff: AI-Driven Cytomorphology Image Synthesis for Medical Diagnostics](https://mlsanity.com/paper/2507.05063/) (2025)
- [SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification](https://mlsanity.com/paper/2605.03301/) (2026)
- [Evaluating the trustworthiness of the Fr'echet Inception Distance with stochastic embedding representations](https://mlsanity.com/paper/2601.21979/) (2026)
- [Ordinal Diffusion Models for Color Fundus Images](https://mlsanity.com/paper/2602.24013/) (2026)
- [Trustworthy deep domain adaptation for wearable photoplethysmography signal analysis with decision-theoretic uncertainty quantification](https://mlsanity.com/paper/2604.17480/) (2026)
- [Diffusion-Based Tumor Inpainting for Renal Segmentation under Clinical Data Scarcity](https://mlsanity.com/paper/2609.21698/) (2026)

## Abstract (arXiv)

Fr'echet Inception Distance (FID), computed with an ImageNet pretrained Inception-v3 network, is widely used as a state-of-the-art evaluation metric for generative models. It assumes that feature vectors from Inception-v3 follow a multivariate Gaussian distribution and calculates the 2-Wasserstein distance based on their means and covariances. While FID effectively measures how closely synthetic data match real data in many image synthesis tasks, the primary goal in biomedical generative models is often to enrich training datasets ideally with corresponding annotations. For this purpose, the gold standard for evaluating generative models is to incorporate synthetic data into downstream task training, such as classification and segmentation, to pragmatically assess its performance. In this paper, we examine cases from retinal imaging modalities, including color fundus photography and optical coherence tomography, where FID and its related metrics misalign with task-specific evaluation goals in classification and segmentation. We highlight the limitations of using various metrics, represented by FID and its variants, as evaluation criteria for these applications and address their potential caveats in broader biomedical imaging modalities and downstream tasks.