# FingER: Content Aware Fine-grained Evaluation with Reasoning for AI-Generated Videos

arXiv: https://arxiv.org/abs/2504.10358 · HTML: https://mlsanity.com/paper/2504.10358/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 26 of 26)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [Open-Sora: Democratizing Efficient Video Production for All](https://mlsanity.com/paper/2412.20404/) (2024)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs](https://mlsanity.com/paper/2406.18629/) (2024)
- [GODIVA: Generating Open-DomaIn Videos from nAtural Descriptions](https://mlsanity.com/paper/2104.14806/) (2021)
- [Unified Reward Model for Multimodal Understanding and Generation](https://mlsanity.com/paper/2503.05236/) (2025)
- [MobileVLM V2: Faster and Stronger Baseline for Vision Language Model](https://mlsanity.com/paper/2402.03766/) (2024)
- [R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model](https://mlsanity.com/paper/2503.05132/) (2025)
- [Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models](https://mlsanity.com/paper/2405.04233/) (2024)
- [VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation](https://mlsanity.com/paper/2412.21059/) (2024)
- [GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning](https://mlsanity.com/paper/2504.02546/) (2025)
- [LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment](https://mlsanity.com/paper/2412.04814/) (2024)
- [Towards A Better Metric for Text-to-Video Generation](https://mlsanity.com/paper/2401.07781/) (2024)
- [VMBench: A Benchmark for Perception-Aligned Video Motion Generation](https://mlsanity.com/paper/2503.10076/) (2025)
- [Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment](https://mlsanity.com/paper/2403.11956/) (2024)
- [The Dawn of Video Generation: Preliminary Explorations with SORA-like Models](https://mlsanity.com/paper/2410.05227/) (2024)
- [Next Token Is Enough: Realistic Image Quality and Aesthetic Scoring with Multimodal Large Language Model](https://mlsanity.com/paper/2503.06141/) (2025)
- [MMGenBench: Fully Automatically Evaluating LMMs from the Text-to-Image Generation Perspective](https://mlsanity.com/paper/2411.14062/) (2024)

## Built on by (most cited first, 22 of 22)

- [UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning](https://mlsanity.com/paper/2505.14231/) (2025)
- [FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editing](https://mlsanity.com/paper/2505.03329/) (2025)
- [DreamX-World 1.0: A General-Purpose Interactive World Model](https://mlsanity.com/paper/2606.16993/) (2026)
- [Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Models](https://mlsanity.com/paper/2508.12880/) (2025)
- [AutoDrive-R2: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Driving](https://mlsanity.com/paper/2509.01944/) (2025)
- [Omni-Effects: Unified and Spatially-Controllable Visual Effects Generation](https://mlsanity.com/paper/2508.07981/) (2025)
- [NarrLV: Towards a Comprehensive Narrative-Centric Evaluation for Long Video Generation](https://mlsanity.com/paper/2507.11245/) (2025)
- [Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models](https://mlsanity.com/paper/2603.22212/) (2026)
- [ImagerySearch: Adaptive Test-Time Search for Video Generation Beyond Semantic Dependency Constraints](https://mlsanity.com/paper/2510.14847/) (2025)
- [MACE-Dance: Motion-Appearance Cascaded Experts for Music-Driven Dance Video Generation](https://mlsanity.com/paper/2512.18181/) (2025)
- [What if Agents Could Imagine? Reinforcing Open-Vocabulary HOI Comprehension through Generation](https://mlsanity.com/paper/2602.11499/) (2026)
- [From Scale to Speed: Adaptive Test-Time Scaling for Image Editing](https://mlsanity.com/paper/2603.00141/) (2026)
- [AIGVE-MACS: Unified Multi-Aspect Commenting and Scoring Model for AI-Generated Video Evaluation](https://mlsanity.com/paper/2507.01255/) (2025)
- [Taming Hallucinations: Boosting MLLMs' Video Understanding via Counterfactual Video Generation](https://mlsanity.com/paper/2512.24271/) (2025)
- [Video-CoE: Reinforcing Video Event Prediction via Chain of Events](https://mlsanity.com/paper/2603.14935/) (2026)
- [R2M-Bench: Evaluating Revisit Memory via Relative Consistency in Interactive Video World Models](https://mlsanity.com/paper/2608.27328/) (2026)
- [End-to-End Multi-Modal Diffusion Mamba](https://mlsanity.com/paper/2510.13253/) (2025)
- [OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Data](https://mlsanity.com/paper/2606.30019/) (2026)
- [SAGA: Source Attribution of Generative AI Videos](https://mlsanity.com/paper/2511.12834/) (2025)
- [IndusAgent: Reinforcing Open-Vocabulary Industrial Anomaly Detection with Agentic Tools](https://mlsanity.com/paper/2605.20682/) (2026)
- [Q-Hawkeye: Reliable Visual Policy Optimization for Image Quality Assessment](https://mlsanity.com/paper/2601.22920/) (2026)
- [Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment](https://mlsanity.com/paper/2607.13941/) (2026)

## Abstract (arXiv)

Recent advances in video generation have posed great challenges in the assessment of AI-generated content, particularly with the emergence of increasingly sophisticated models. The various inconsistencies and defects observed in such videos are inherently complex, making overall scoring notoriously difficult. In this paper, we emphasize the critical importance of integrating fine-grained reasoning into video evaluation, and we propose FingER, a novel entity-level reasoning evaluation framework that first automatically generates Fine-grained Entity-level questions, and then answers those questions by a Reasoning model with scores, which can be subsequently weighted summed to an overall score for different applications. Specifically, we leverage LLMs to derive entity-level questions across five distinct perspectives, which (i) often focus on some specific entities of the content, thereby making answering or scoring much easier by MLLMs, and (ii) are more interpretable. Then we construct a FingER dataset, consisting of approximately 3.3k videos and corresponding 60k fine-grained QA annotations, each with detailed reasons. Based on that, we further investigate various training protocols to best incentivize the reasoning capability of MLLMs for correct answer prediction. Extensive experiments demonstrate that a reasoning model trained using Group Relative Policy Optimization (GRPO) with a cold-start strategy achieves the best performance. Notably, our model surpasses existing methods by a relative margin of 11.8% on GenAI-Bench and 5.5% on MonetBench with only 3.3k training videos, which is at most one-tenth of the training samples utilized by other methods. Our code and dataset will be released soon.