mlsanityData to 7 Oct 2026

VepAgent: Bridging Causal-Transition via Tool-Augmented Reinforcement Learning for Video Event Prediction

arXiv 2610.06293 ↗First posted 5 Oct 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated remarkable potential in video understanding, yet their reliance on retrospective summarization and text-centric priors often limits their ability to bridge unobserved causal transitions when applied to Video Event Prediction (VEP). To address this, we propose VepAgent, an agentic framework that integrates causal-transition reasoning with tool-augmented reinforcement learning (RL) for robust VEP. Unlike prior methods that passively project future trajectories from historical dependencies, our approach explicitly models the logical progression from terminal observed states to future events. Specifically, we first construct futurebench-4K, a high-quality chain-of-thought dataset for supervised fine-tuning (SFT) that effectively bridges the causal-logic gap by structuring the deduction of unobserved intermediate states. Subsequently, we develop a diagnostic tool library integrating state tracking, frame retrieval, and region magnification, enabling the agent to dynamically augment reasoning with external tools to recover missing spatio-temporal evidence and resolve visual ambiguities during inference. Moreover, we propose a composite reward mechanism that jointly optimizes prediction accuracy, causal coherence, and reliable prior, compelling the agent to rely on genuine visual grounding rather than superficial textual similarities. Extensive evaluations on FutureBench and NEPBench datasets demonstrate that our method achieves state-of-the-art performance, significantly outperforming larger MLLMs and validating the empirical effectiveness of our agentic, future-oriented reasoning paradigm.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. First posted on arXiv 5 Oct 2026.

Builds on

No reference list yet.

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Use
  2. 2026GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking
  3. 2026Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction
  4. 2025What Happens Next? Next Scene Prediction with a Unified Video Model
  5. 2025VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models
  6. 2026TimeThink: Reasoning with Time for Video LLMs
  7. 2025MOSS-ChatV: Reinforcement Learning with Process Reasoning Reward for Video Temporal Reasoning
  8. 2026Agentic Video Understanding: A Survey