# An Improved Evaluation Framework for Generative Adversarial Networks

arXiv: https://arxiv.org/abs/1803.07474 · HTML: https://mlsanity.com/paper/1803.07474/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2018.

## Built on by (most cited first, 26 of 26)

- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [Why Tabular Foundation Models Should Be a Research Priority](https://mlsanity.com/paper/2405.01147/) (2024)
- [Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2304.01816/) (2023)
- [RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model](https://mlsanity.com/paper/2309.00810/) (2023)
- [StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesis](https://mlsanity.com/paper/2206.09479/) (2022)
- [Time Weaver: A Conditional Time Series Generation Model](https://mlsanity.com/paper/2403.02682/) (2024)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [UnMarker: A Universal Attack on Defensive Image Watermarking](https://mlsanity.com/paper/2405.08363/) (2024)
- [Conditional Frechet Inception Distance](https://mlsanity.com/paper/2103.11521/) (2021)
- [DALLE-URBAN: Capturing the urban design expertise of large text to image transformers](https://mlsanity.com/paper/2208.04139/) (2022)
- [Influence Estimation for Generative Adversarial Networks](https://mlsanity.com/paper/2101.08367/) (2021)
- [Evaluating generative networks using Gaussian mixtures of image features](https://mlsanity.com/paper/2110.05240/) (2021)
- [Quantum Denoising Diffusion Models](https://mlsanity.com/paper/2401.07049/) (2024)
- [A Pragmatic Note on Evaluating Generative Models with Fr'echet Inception Distance for Retinal Image Synthesis](https://mlsanity.com/paper/2502.17160/) (2025)
- [Edge-preserving noise for diffusion models](https://mlsanity.com/paper/2410.01540/) (2024)
- [LatentQGAN: A Hybrid QGAN with Classical Convolutional Autoencoder](https://mlsanity.com/paper/2409.14622/) (2024)
- [Multi-domain stain normalization for digital pathology: A cycle-consistent adversarial network for whole slide images](https://mlsanity.com/paper/2301.09431/) (2023)
- [Let's Get the FACS Straight -- Reconstructing Obstructed Facial Features](https://mlsanity.com/paper/2311.05221/) (2023)
- [The Challenges of Image Generation Models in Generating Multi-Component Images](https://mlsanity.com/paper/2311.13620/) (2023)
- [Can representation learning for multimodal image registration be improved by supervision of intermediate layers?](https://mlsanity.com/paper/2303.00403/) (2023)
- [Mode-Aware Continual Learning for Conditional Generative Adversarial Networks](https://mlsanity.com/paper/2305.11400/) (2023)
- [Deceptive Diffusion: Generating Synthetic Adversarial Examples](https://mlsanity.com/paper/2406.19807/) (2024)
- [Generative Models with Information-Theoretic Protection Against Membership Inference Attacks](https://mlsanity.com/paper/2206.00071/) (2022)
- [Offline Evaluation of Set-Based Text-to-Image Generation](https://mlsanity.com/paper/2410.17331/) (2024)
- [Using Skew to Assess the Quality of GAN-generated Image Features](https://mlsanity.com/paper/2310.20636/) (2023)
- [HYDRO: Towards Non-Reversible Face De-Identification Using a High-Fidelity Hybrid Diffusion and Target-Oriented Approach](https://mlsanity.com/paper/2609.27011/) (2026)

## Abstract (arXiv)

In this paper, we propose an improved quantitative evaluation framework for Generative Adversarial Networks (GANs) on generating domain-specific images, where we improve conventional evaluation methods on two levels: the feature representation and the evaluation metric. Unlike most existing evaluation frameworks which transfer the representation of ImageNet inception model to map images onto the feature space, our framework uses a specialized encoder to acquire fine-grained domain-specific representation. Moreover, for datasets with multiple classes, we propose Class-Aware Frechet Distance (CAFD), which employs a Gaussian mixture model on the feature space to better fit the multi-manifold feature distribution. Experiments and analysis on both the feature level and the image level were conducted to demonstrate improvements of our proposed framework over the recently proposed state-of-the-art FID method. To our best knowledge, we are the first to provide counter examples where FID gives inconsistent results with human judgments. It is shown in the experiments that our framework is able to overcome the shortness of FID and improves robustness. Code will be made available.