# VideoGAIA: A Benchmark for General AI Assistants on Agentic Video Understanding

arXiv: https://arxiv.org/abs/2608.14718 · HTML: https://mlsanity.com/paper/2608.14718/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 12 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Video understanding with MLLMs](https://mlsanity.com/mllm/video-understanding-with-mllms/)
- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 28 of 28)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024)
- [VideoAgent: Long-form Video Understanding with Large Language Model as Agent](https://mlsanity.com/paper/2403.10517/) (2024)
- [VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding](https://mlsanity.com/paper/2403.11481/) (2024)
- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding](https://mlsanity.com/paper/2406.14515/) (2024)
- [Thyme: Think Beyond Images](https://mlsanity.com/paper/2508.11630/) (2025)
- [MMVU: Measuring Expert-Level Multi-Discipline Video Understanding](https://mlsanity.com/paper/2501.12380/) (2025)
- [WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent](https://mlsanity.com/paper/2508.05748/) (2025)
- [Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding](https://mlsanity.com/paper/2505.18079/) (2025)
- [Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity](https://mlsanity.com/paper/2607.00248/) (2026)
- [VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks](https://mlsanity.com/paper/2410.19100/) (2024)
- [Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2601.22060/) (2026)
- [GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents](https://mlsanity.com/paper/2604.26752/) (2026)
- [VideoChat-A1: Thinking with Long Videos by Chain-of-Shot Reasoning](https://mlsanity.com/paper/2506.06097/) (2025)
- [OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents](https://mlsanity.com/paper/2605.05185/) (2026)
- [MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models](https://mlsanity.com/paper/2508.09210/) (2025)
- [Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoning](https://mlsanity.com/paper/2601.06943/) (2026)
- [Video-Browser: Towards Agentic Open-web Video Browsing](https://mlsanity.com/paper/2512.23044/) (2025)
- [VideoThinker: Building Agentic VideoLLMs with LLM-Guided Tool Reasoning](https://mlsanity.com/paper/2601.15724/) (2026)
- [LongVidSearch: An Agentic Benchmark for Multi-hop Evidence Retrieval Planning in Long Videos](https://mlsanity.com/paper/2603.14468/) (2026)
- [Orchestra-o1: Omnimodal Agent Orchestration](https://mlsanity.com/paper/2606.13707/) (2026)

## Abstract (arXiv)

Video understanding is a fundamental task for evaluating the capabilities of multimodal large language models (MLLMs). However, existing leading models have already achieved approximately 90% accuracy on the Video-MME leaderboard, suggesting that conventional single-turn video understanding tasks are becoming increasingly saturated and insufficient for assessing the intelligence of advanced MLLMs. Towards this end, we introduce VideoGAIA, an agentic video understanding benchmark for general artificial intelligence (AI) assistants. Moving beyond one-shot video question answering, VideoGAIA formulates video understanding as a multi-turn, tool-augmented interaction process, where models must iteratively perceive videos, invoke external tools, gather complementary information, and integrate multimodal evidence across turns. VideoGAIA contains 271 model-human co-designed tasks covering diverse and complex real-world scenarios. Each video-question-answer instance is independently verified by three human experts to ensure both correctness and appropriate difficulty. All evaluated MLLMs, including frontier models such as GPT-5.5 and Kimi-K3, achieve less than 60% accuracy on VideoGAIA, highlighting its value as a high-quality and timely benchmark for evaluating next-generation MLLMs. We hope that VideoGAIA will facilitate the transition from conventional video understanding toward agentic video understanding.