# Evaluating Design Video Generation: Metrics for Compositional Fidelity

arXiv: https://arxiv.org/abs/2605.16223 · HTML: https://mlsanity.com/paper/2605.16223/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 15 May 2026.

## Builds on (most cited first, 16 of 16)

- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [YOLOv11: An Overview of the Key Architectural Enhancements](https://mlsanity.com/paper/2410.17725/) (2024)
- [EvalCrafter: Benchmarking and Evaluating Large Video Generation Models](https://mlsanity.com/paper/2310.11440/) (2023)
- [CanvasVAE: Learning to Generate Vector Graphic Documents](https://mlsanity.com/paper/2108.01249/) (2021)
- [PosterLayout: A New Benchmark and Approach for Content-aware Visual-Textual Presentation Layout](https://mlsanity.com/paper/2303.15937/) (2023)
- [Composition-aware Graphic Layout GAN for Visual-textual Presentation Designs](https://mlsanity.com/paper/2205.00303/) (2022)
- [PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a Unified Framework](https://mlsanity.com/paper/2506.10741/) (2025)
- [LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer](https://mlsanity.com/paper/2212.09877/) (2022)
- [AdsQA: Towards Advertisement Video Understanding](https://mlsanity.com/paper/2509.08621/) (2025)
- [OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilities](https://mlsanity.com/paper/2507.15085/) (2025)
- [VideoAds for Fast-Paced Video Understanding](https://mlsanity.com/paper/2504.09282/) (2025)
- [Graphic-Design-Bench: A Comprehensive Benchmark for Evaluating AI on Graphic Design Tasks](https://mlsanity.com/paper/2604.04192/) (2026)
- [LICA: Layered Image Composition Annotations for Graphic Design Research](https://mlsanity.com/paper/2603.16098/) (2026)
- [Can Vision Language Models Assess Graphic Design Aesthetics? A Benchmark, Evaluation, and Dataset Perspective](https://mlsanity.com/paper/2603.01083/) (2026)
- [E-VAds: An E-commerce Short Videos Understanding Benchmark for MLLMs](https://mlsanity.com/paper/2602.08355/) (2026)

## Abstract (arXiv)

Generative video models are increasingly used in design animation tasks, yet no standardized evaluation framework exists for this domain. Unlike natural video generation, design animation imposes structured constraints: specific components shall animate with prescribed motion types, directions, speed and timing, while non-animated regions must remain stable and layout structure must be preserved. This paper provides a fully automated evaluation framework organized across four dimensions: layout fidelity, motion correctness, temporal quality, and content fidelity. This eliminates the reliance on subjective human evaluation and establishes a common basis for benchmarking progress in the field. We release the code and dataset here: https://github.com/purvanshi/lica-bench.