VideoEvolve: Evolving Agent Harnesses for Video Temporal Grounding
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Video temporal grounding aims to localize events in videos from natural-language queries. For agents built around frozen video-language models, the harness determines how queries guide temporal predictions and how those predictions are refined. Manually refining these harnesses requires diagnosing grounding failures and coordinating changes to both agent workflows and instructions. We introduce VideoEvolve, a framework that automatically evolves agent harnesses for video temporal grounding. VideoEvolve uses a Cloze-Structured Harness Representation that preserves stage interfaces while leaving agent workflows and instructions open to evolution. Branch-Guided Harness Evolution preserves promising code branches for continued refinement, using execution feedback to guide local edits and validation to determine which improvements are carried forward. Experiments demonstrate improved grounding performance across multiple benchmarks. Component analyses identify instruction refinement as a consistent source of gains, while the benefits of evolved code vary across evaluation settings. Together, these results support automated harness evolution as an effective approach to improving video temporal grounding. Code is available at https://github.com/bingjunluo/VideoEvolve .
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 17 of its 18 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
17 of 18 references.
Closest ancestors · 1
Same direction, newest first
Other references · 16
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
- 2024Automated Design of Agentic Systemscited by 360
- 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
- 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
- 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
- 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
- 2026Self-Harness: Harnesses That Improve Themselvescited by 79
- 2026AutoHarness: improving LLM agents by automatically synthesizing a code harnesscited by 63
- 2024ChatVTG: Video Temporal Grounding via Chat with Video Dialogue Large Language Modelscited by 54
- 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 46
- 2023MH-DETR: Video Moment and Highlight Detection with Cross-modal Transformercited by 40
4 more
- 2024Training-free Video Temporal Grounding using Large-scale Pre-trained Modelscited by 28
- 2024LLaVA-MR: Large Language-and-Vision Assistant for Video Moment Retrievalcited by 17
- 2026EvoGround: Self-Evolving Video Agents for Video Temporal Groundingcited by 3
- 2026DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Groundingcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CoEvoWhen: Policy-Tool Coevolution for Ultra-Long Video Temporal Grounding
- 2026VidHarness: Evolving Agent Harnesses for Cost-Efficient Long Video Understanding
- 2026Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
- 2026VideoWeaver: Evaluating and Evolving Skills for Agentic Long Video Generation
- 2026Conditional Multi-Event Temporal Grounding in Long-Form Video
- 2025Datasets and Recipes for Video Temporal Grounding via Reinforcement Learning
- 2025Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Temporal Grounding
- 2025VideoTG-R1: Boosting Video Temporal Grounding via Curriculum Reinforcement Learning on Reflected Boundary Annotations