# Beyond the Last Frame: Process-aware Evaluation for Generative Video Reasoning

arXiv: https://arxiv.org/abs/2512.24952 · HTML: https://mlsanity.com/paper/2512.24952/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 31 Dec 2025.

## Builds on (most cited first, 19 of 19)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://mlsanity.com/paper/2402.14804/) (2024)
- [Pyramidal Flow Matching for Efficient Video Generative Modeling](https://mlsanity.com/paper/2410.05954/) (2024)
- [Video models are zero-shot learners and reasoners](https://mlsanity.com/paper/2509.20328/) (2025)
- [Are Video Models Ready as Zero-Shot Reasoners? An Empirical Study with the MME-CoF Benchmark](https://mlsanity.com/paper/2510.26802/) (2025)
- [Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm](https://mlsanity.com/paper/2511.04570/) (2025)
- [Pathways on the Image Manifold: Image Editing via Video Generation](https://mlsanity.com/paper/2411.16819/) (2024)
- [V-ReasonBench: Toward Unified Reasoning Benchmark Suite for Video Generation Models](https://mlsanity.com/paper/2511.16668/) (2025)
- [TiViBench: Benchmarking Think-in-Video Reasoning for Video Generative Models](https://mlsanity.com/paper/2511.13704/) (2025)
- [Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmark](https://mlsanity.com/paper/2511.13853/) (2025)
- [Video Models Start to Solve Chess, Maze, Sudoku, Mental Rotation, and Raven' Matrices](https://mlsanity.com/paper/2512.05969/) (2025)

## Built on by (most cited first, 4 of 4)

- [Video Models Reason Early: Exploiting Plan Commitment for Maze Solving](https://mlsanity.com/paper/2603.30043/) (2026)
- [CollabVR: Collaborative Video Reasoning with Vision-Language and Video Generation Models](https://mlsanity.com/paper/2605.08735/) (2026)
- [Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence](https://mlsanity.com/paper/2607.16401/) (2026)
- [Thinking in Video: Can Video Generators Really Reason About the Real World?](https://mlsanity.com/paper/2607.17523/) (2026)

## Abstract (arXiv)

Recent breakthroughs in video generation have demonstrated an emerging capability termed Chain-of-Frames (CoF) reasoning, where models resolve complex tasks through the generation of continuous frames. While these models show promise for Generative Video Reasoning (GVR), existing evaluation frameworks often rely on single-frame assessments, which can lead to outcome-hacking, where a model reaches a correct conclusion through an erroneous process. To address this, we propose a process-aware evaluation paradigm. We introduce VIPER, a comprehensive benchmark spanning 16 tasks across temporal, structural, symbolic, spatial, physics, and planning reasoning. Furthermore, we propose Process-outcome Consistency (POC@r), a new metric that utilizes VLM-as-Judge with a hierarchical rubric to evaluate both the validity of the intermediate steps and the final result. Our experiments reveal that state-of-the-art video models achieve POC@1.0 only about 20% and exhibit a significant outcome-hacking. We further explore the impact of test-time scaling and sampling robustness, highlighting a substantial gap between current video generation and true generalized visual reasoning. Our benchmark are released at https://github.com/RUCAIBox/VIPER.