# MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understanding

arXiv: https://arxiv.org/abs/2608.04587 · HTML: https://mlsanity.com/paper/2608.04587/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Builds on (most cited first, 22 of 22)

- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [AFlow: Automating Agentic Workflow Generation](https://mlsanity.com/paper/2410.10762/) (2024)
- [Automated Design of Agentic Systems](https://mlsanity.com/paper/2408.08435/) (2024)
- [VideoAgent: Long-form Video Understanding with Large Language Model as Agent](https://mlsanity.com/paper/2403.10517/) (2024)
- [VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos](https://mlsanity.com/paper/2405.19209/) (2024)
- [EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms](https://mlsanity.com/paper/2406.14228/) (2024)
- [Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding](https://mlsanity.com/paper/2505.18079/) (2025)
- [Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory](https://mlsanity.com/paper/2508.09736/) (2025)
- [CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding](https://mlsanity.com/paper/2412.12075/) (2024)
- [Attentive Eraser: Unleashing Diffusion Model's Object Removal Potential via Self-Attention Redirection Guidance](https://mlsanity.com/paper/2412.12974/) (2024)
- [WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning](https://mlsanity.com/paper/2512.02425/) (2025)
- [VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning](https://mlsanity.com/paper/2503.13444/) (2025)
- [RAMS-Trans: Recurrent Attention Multi-scale Transformer forFine-grained Image Recognition](https://mlsanity.com/paper/2107.08192/) (2021)
- [OmAgent: A Multi-modal Agent Framework for Complex Video Understanding with Task Divide-and-Conquer](https://mlsanity.com/paper/2406.16620/) (2024)
- [VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understanding](https://mlsanity.com/paper/2512.12360/) (2025)
- [Erase Diffusion: Empowering Object Removal Through Calibrating Diffusion Pathways](https://mlsanity.com/paper/2503.07026/) (2025)
- [Diffusion Probe: Generated Image Result Prediction Using CNN Probes](https://mlsanity.com/paper/2602.23783/) (2026)
- [Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video Understanding](https://mlsanity.com/paper/2603.17307/) (2026)
- [Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts](https://mlsanity.com/paper/2604.08541/) (2026)
- [Hierarchical Long Video Understanding with Audiovisual Entity Cohesion and Agentic Search](https://mlsanity.com/paper/2601.13719/) (2026)
- [simpleposter: A simple baseline for product poster generation](https://mlsanity.com/paper/2605.08784/) (2026)
- [Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning](https://mlsanity.com/paper/2607.01191/) (2026)

## Built on by (most cited first, 3 of 3)

- [VideoHarness-RSI: Recursive Harness Self-Improvement for Long-Video Understanding with Frozen Vision-Language Models](https://mlsanity.com/paper/2608.24302/) (2026)
- [VidHarness: Evolving Agent Harnesses for Cost-Efficient Long Video Understanding](https://mlsanity.com/paper/2609.38413/) (2026)
- [VideoEvoHarness: Self Evolving Agent Harness from Unlabeled Videos for Video Understanding](https://mlsanity.com/paper/2610.15389/) (2026)

## Abstract (arXiv)

Long-form video understanding requires locating sparse, question-relevant evidence in long, multimodal videos. Real-world video distributions differ in modality-specific information density, content structure, and evidence patterns, causing fixed video-agent designs to incur redundant processing or fail when mismatched. Extending automated agent evolution from text to video is challenging because full long-video execution makes candidate validation expensive, failures propagate across coupled evidence-processing stages, and complex preprocessing, perception tools, and localization strategies make code-level updates difficult to implement reliably. We introduce MetaVideoAgent, a framework that automatically evolves a video agent for a target distribution. It profiles information density and evidence requirements from sparsely sampled frames and associated queries to guide initial design, then compresses localized failures into independently executable minimal validation tasks. It constructs evidence-grounded Gold Paths, audits Student trajectories, aggregates recurring failures across samples, and attributes them to responsible modules. A modular agent representation constrains each update to the primary responsible module and its necessary dependencies. We further introduce VA-EvoBench, covering eight video distributions with separate evolution and held-out splits. With four evolution iterations per distribution, MetaVideoAgent improves every initial agent and raises macro-average accuracy from 38.44% to 51.47%, at an average evolution cost of 3.54M tokens per distribution. The evolved agents outperform the strongest prior fixed-design video agent by 6.39 percentage points while using the fewest tokens and video frames per question among the compared video agents. We will release all code and data to support reproducible research.