# VGI-Bench: Probing Visual Intelligence in Video Generation Models

arXiv: https://arxiv.org/abs/2608.19583 · HTML: https://mlsanity.com/paper/2608.19583/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 20 Aug 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 28 of 28)

- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Video models are zero-shot learners and reasoners](https://mlsanity.com/paper/2509.20328/) (2025)
- [Seedance 2.0: Advancing Video Generation for World Complexity](https://mlsanity.com/paper/2604.14148/) (2026)
- [DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos](https://mlsanity.com/paper/2602.06949/) (2026)
- [Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing](https://mlsanity.com/paper/2504.02826/) (2025)
- [Video Generators are Robot Policies](https://mlsanity.com/paper/2508.00795/) (2025)
- [UniVideo: Unified Understanding, Generation, and Editing for Videos](https://mlsanity.com/paper/2510.08377/) (2025)
- [Vid2World: Crafting Video Diffusion Models to Interactive World Models](https://mlsanity.com/paper/2505.14357/) (2025)
- [KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Models](https://mlsanity.com/paper/2505.16707/) (2025)
- [Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm](https://mlsanity.com/paper/2511.04570/) (2025)
- [V-ReasonBench: Toward Unified Reasoning Benchmark Suite for Video Generation Models](https://mlsanity.com/paper/2511.16668/) (2025)
- [When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought](https://mlsanity.com/paper/2511.02779/) (2025)
- [TiViBench: Benchmarking Think-in-Video Reasoning for Video Generative Models](https://mlsanity.com/paper/2511.13704/) (2025)
- [Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmark](https://mlsanity.com/paper/2511.13853/) (2025)
- [DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models](https://mlsanity.com/paper/2512.24165/) (2025)
- [Video Generation Models are General-Purpose Vision Learners](https://mlsanity.com/paper/2607.09024/) (2026)
- [RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligence](https://mlsanity.com/paper/2512.02622/) (2025)
- [Video Models Reason Early: Exploiting Plan Commitment for Maze Solving](https://mlsanity.com/paper/2603.30043/) (2026)
- [Video Models Can Reason with Verifiable Rewards](https://mlsanity.com/paper/2605.15458/) (2026)
- [MentisOculi: Revealing the Limits of Reasoning with Mental Imagery](https://mlsanity.com/paper/2602.02465/) (2026)
- [MORSE-500: A Programmatically Controllable Video Benchmark to Stress-Test Multimodal Reasoning](https://mlsanity.com/paper/2506.05523/) (2025)
- [EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models](https://mlsanity.com/paper/2603.12252/) (2026)
- [WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors](https://mlsanity.com/paper/2605.10434/) (2026)
- [MME-CoF-Pro: Evaluating Reasoning Coherence in Video Generative Models with Text and Visual Hints](https://mlsanity.com/paper/2603.20194/) (2026)
- [VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization](https://mlsanity.com/paper/2606.02564/) (2026)
- [CollabVR: Collaborative Video Reasoning with Vision-Language and Video Generation Models](https://mlsanity.com/paper/2605.08735/) (2026)
- [V-Bridge: Bridging Video Generative Priors to Versatile Few-shot Image Restoration](https://mlsanity.com/paper/2603.13089/) (2026)
- [OpenCoF: Learning to Reason Through Video Generation](https://mlsanity.com/paper/2607.08763/) (2026)

## Built on by (most cited first, 3 of 3)

- [VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning](https://mlsanity.com/paper/2608.26105/) (2026)
- [ProAR: Learning Prospective Reasoning with Autoregressive Video Models](https://mlsanity.com/paper/2610.03664/) (2026)
- [OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning](https://mlsanity.com/paper/2610.12458/) (2026)

## Abstract (arXiv)

Recent studies suggest that video generation models can exhibit certain forms of zero-shot visual reasoning through generated frames. Yet reliable evaluation remains challenging: benchmarks should adopt inputs aligned with the visual priors of current video models, require valid evolving processes rather than only plausible final states, and calibrate task difficulty to remain challenging yet partly feasible. To this end, we introduce VGI-bench, containing 27 tasks and 810 instances, organized by a two-level taxonomy of task domains and skill tags for fine-grained evaluation of visual reasoning capabilities of video generation models. Our evaluations show that current generative systems can solve a subset of visually grounded reasoning tasks, but remain far from reliable, with even the strongest model, Seedance 2.0, achieving only 51.0% under our evaluation criteria. Our analysis further explore the output failure modes, input condition sensitivity, performance transfer boundary from synthetic fine-tuning, and internal denoising perspective revealing limited self-correction, where later steps mainly refine early hypotheses rather than correct reasoning errors. We hope VGI-bench will help stimulate the development of next-generation video generation models. Website: https://hexuan21.github.io/VGI-Bench/