# Beyond FVD: Enhanced Evaluation Metrics for Video Generation Quality

arXiv: https://arxiv.org/abs/2410.05203 · HTML: https://mlsanity.com/paper/2410.05203/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 77 references that are in the map.
- First posted on arXiv 7 Oct 2024.

## Builds on (most cited first, 30 of 35)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2212.11565/) (2022)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [Latent Video Diffusion Models for High-Fidelity Long Video Generation](https://mlsanity.com/paper/2211.13221/) (2022)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)
- [Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2309.15818/) (2023)
- [StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2](https://mlsanity.com/paper/2112.14683/) (2021)
- [N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtion](https://mlsanity.com/paper/2111.12417/) (2021)
- [GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium](https://mlsanity.com/paper/1706.08500/) (2017)
- [The Role of ImageNet Classes in Fr'echet Inception Distance](https://mlsanity.com/paper/2203.06026/) (2022)
- [How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language](https://mlsanity.com/paper/2008.08143/) (2020)
- [Effectively Unbiased FID and Inception Score and where to find them](https://mlsanity.com/paper/1911.07023/) (2019)
- [Self-Supervised Visual Planning with Temporal Skip Connections](https://mlsanity.com/paper/1710.05268/) (2017)
- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [DwNet: Dense warp-based network for pose-guided human video generation](https://mlsanity.com/paper/1910.09139/) (2019)
- [Adapting Frechet Audio Distance for Generative Music Evaluation](https://mlsanity.com/paper/2311.01616/) (2023)
- [Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks](https://mlsanity.com/paper/1709.07592/) (2017)

## Built on by (most cited first, 22 of 22)

- [WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models](https://mlsanity.com/paper/2602.08971/) (2026)
- [Mobile Video Diffusion](https://mlsanity.com/paper/2412.07583/) (2024)
- [AKiRa: Augmentation Kit on Rays for optical video generation](https://mlsanity.com/paper/2412.14158/) (2024)
- [SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models](https://mlsanity.com/paper/2605.23345/) (2026)
- [Direct Motion Models for Assessing Generated Videos](https://mlsanity.com/paper/2505.00209/) (2025)
- [ConsID-Gen: View-Consistent and Identity-Preserving Image-to-Video Generation](https://mlsanity.com/paper/2602.10113/) (2026)
- [What Are You Doing? A Closer Look at Controllable Human Video Generation](https://mlsanity.com/paper/2503.04666/) (2025)
- [What Drives Compositional Generalization in Visual Generative Models? The Importance of Continuous Training Objectives](https://mlsanity.com/paper/2510.03075/) (2025)
- [World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications](https://mlsanity.com/paper/2606.00133/) (2026)
- [CaliBench: Are the Stochastic Dynamics of Video World Models Physically Calibrated?](https://mlsanity.com/paper/2608.16829/) (2026)
- [Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling](https://mlsanity.com/paper/2504.05537/) (2025)
- [Measuring 3D Spatial Geometric Consistency in Dynamic Video Generation](https://mlsanity.com/paper/2603.19048/) (2026)
- [Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators](https://mlsanity.com/paper/2607.07196/) (2026)
- [TextOCVP: Object-Centric Video Prediction with Language Guidance](https://mlsanity.com/paper/2502.11655/) (2025)
- [RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction](https://mlsanity.com/paper/2608.28718/) (2026)
- [Evaluating Text-to-Image and Text-to-Video Synthesis with a Conditional Fr'echet Distance](https://mlsanity.com/paper/2503.21721/) (2025)
- [Low-Bit, High-Fidelity: Optimal Transport Quantization for Flow Matching](https://mlsanity.com/paper/2511.11418/) (2025)
- [Physics-Guided Diffusion Priors for Multi-Slice Reconstruction in Scientific Imaging](https://mlsanity.com/paper/2512.06977/) (2025)
- [UniVR: Thinking in Visual Space for Unified Visual Reasoning](https://mlsanity.com/paper/2607.12800/) (2026)
- [SlotDiT: Object-Centric Representations for Diffusion Transformers](https://mlsanity.com/paper/2609.17414/) (2026)
- [CtrlWAM: Controllable World Action Models with Aligned Intent and Foresight](https://mlsanity.com/paper/2610.00859/) (2026)
- [Kuration SDK: Addressing the Virtual2Real Gap via Data Curation](https://mlsanity.com/paper/2610.09305/) (2026)

## Abstract (arXiv)

The Fr'echet Video Distance (FVD) is a widely adopted metric for evaluating video generation distribution quality. However, its effectiveness relies on critical assumptions. Our analysis reveals three significant limitations: (1) the non-Gaussianity of the Inflated 3D Convnet (I3D) feature space; (2) the insensitivity of I3D features to temporal distortions; (3) the impractical sample sizes required for reliable estimation. These findings undermine FVD's reliability and show that FVD falls short as a standalone metric for video generation evaluation. After extensive analysis of a wide range of metrics and backbone architectures, we propose JEDi, the JEPA Embedding Distance, based on features derived from a Joint Embedding Predictive Architecture, measured using Maximum Mean Discrepancy with polynomial kernel. Our experiments on multiple open-source datasets show clear evidence that it is a superior alternative to the widely used FVD metric, requiring only 16% of the samples to reach its steady value, while increasing alignment with human evaluation by 34%, on average.