# Evaluating the trustworthiness of the Fr'echet Inception Distance with stochastic embedding representations

arXiv: https://arxiv.org/abs/2601.21979 · HTML: https://mlsanity.com/paper/2601.21979/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 17 of 17)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [The Role of ImageNet Classes in Fr'echet Inception Distance](https://mlsanity.com/paper/2203.06026/) (2022)
- [Modeling Uncertainty with Hedged Instance Embedding](https://mlsanity.com/paper/1810.00319/) (2018)
- [Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review](https://mlsanity.com/paper/2307.13125/) (2023)
- [How Transferable Are Self-supervised Features in Medical Image Classification Tasks?](https://mlsanity.com/paper/2108.10048/) (2021)
- [Feature Extraction for Generative Medical Imaging Evaluation: New Evidence Against an Evolving Trend](https://mlsanity.com/paper/2311.13717/) (2023)
- [Fr'echet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasets](https://mlsanity.com/paper/2412.01496/) (2024)
- [Metrics that matter: Evaluating image quality metrics for medical image generation](https://mlsanity.com/paper/2505.07175/) (2025)
- [Stochastic Prototype Embeddings](https://mlsanity.com/paper/1909.11702/) (2019)
- [A Pragmatic Note on Evaluating Generative Models with Fr'echet Inception Distance for Retinal Image Synthesis](https://mlsanity.com/paper/2502.17160/) (2025)
- [Accurate super-resolution low-field brain MRI](https://mlsanity.com/paper/2202.03564/) (2022)
- [Quantifying the uncertainty of model-based synthetic image quality metrics](https://mlsanity.com/paper/2504.03623/) (2025)

## Abstract (arXiv)

Feature embeddings acquired from pretrained models are widely used in medical applications of deep learning to assess the characteristics of datasets; e.g. to determine the quality of synthetic, generated medical images. The Fr'echet Inception Distance (FID) is one popular synthetic image quality metric that relies on the assumption that the characteristic features of the data can be detected and encoded by an InceptionV3 model pretrained on ImageNet1K (natural images). While it is widely known that this makes it less effective for applications involving medical images, the extent to which the metric fails to capture meaningful differences in image characteristics is not obviously known. Here, we use Monte Carlo dropout to compute the predictive variance in the FID as well as a supplemental estimate of the predictive variance in the feature embedding model's latent representations. We show that the magnitudes of the predictive variances considered exhibit varying degrees of correlation with the extent to which test inputs (ImageNet1K validation set augmented at various strengths, and other external datasets) are out-of-distribution relative to its training data, providing some insight into the effectiveness of their use as indicators of the trustworthiness of the FID.