UniVA: Universal Video Agent towards Open-Source Next-Generation Video Generalist
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
While specialized AI models excel at isolated video tasks like generation or understanding, real-world applications demand complex, iterative workflows that combine these capabilities. To bridge this gap, we introduce UniVA, an open-source, omni-capable multi-agent framework for next-generation video generalists that unifies video understanding, segmentation, editing, and generation into cohesive workflows. UniVA employs a Plan-and-Act dual-agent architecture that drives a highly automated and proactive workflow: a planner agent interprets user intentions and decomposes them into structured video-processing steps, while executor agents execute these through modular, MCP-based tool servers (for analysis, generation, editing, tracking, etc.). Through a hierarchical multi-level memory (global knowledge, task context, and user-specific preferences), UniVA sustains long-horizon reasoning, contextual continuity, and inter-agent communication, enabling interactive and self-reflective video creation with full traceability. This design enables iterative and any-conditioned video workflows (e.g., text/image/video-conditioned generation rightarrow multi-round editing rightarrow object segmentation rightarrow compositional synthesis) that were previously cumbersome to achieve with single-purpose models or monolithic video-language models. We also introduce UniVA-Bench, a benchmark suite of multi-step video tasks spanning understanding, editing, segmentation, and generation, to rigorously evaluate such agentic video systems. Both UniVA and UniVA-Bench are fully open-sourced, aiming to catalyze research on interactive, agentic, and general-purpose video intelligence for the next generation of multimodal AI systems. (https://univa.online/)
Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 53 of its 60 references that are in the map. First posted on arXiv 11 Nov 2025.
Builds on
53 of 60 references.
Closest ancestors · 5
Same direction, newest first
- 2025PreMind: Multi-Agent Video Understanding for Advanced Indexing of Presentation-style Videos28 Feb
- 2024Enhancing Video-LLM Reasoning via Agent-of-Thoughts Distillation2 Dec
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding18 Mar
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agent15 Mar
- 2024LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing15 Feb
Other references · 48
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
- 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
- 2023A Survey on Multimodal Large Language Modelscited by 898
- 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
- 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
36 more
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
- 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
- 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
- 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
- 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
- 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 200
- 2023Tracking Anything with Decoupled Video Segmentationcited by 197
- 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
- 2020Hierarchical Conditional Relation Networks for Video Question Answeringcited by 163
- 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 154
- 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 153
- 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 151
- 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
- 2025A Survey of Context Engineering for Large Language Modelscited by 120
- 2022Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?cited by 90
- 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 84
- 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
- 2024Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editingcited by 49
- 2025MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encodercited by 44
- 2022Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Groundingcited by 40
- 2025Omni-Video: Democratizing Unified Video Understanding and Generationcited by 39
- 2025ThinkSound: Chain-of-Thought Reasoning in Multimodal Large Language Models for Audio Generation and Editingcited by 33
- 2024StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaborationcited by 28
- 2024VideoAgent: Self-Improving Video Generationcited by 24
- 2014A Discriminative CNN Video Representation for Event Detectioncited by 20
- 2025On Path to Multimodal Generalist: General-Level and General-Benchcited by 18
- 2025VideoMultiAgents: A Multi-Agent Framework for Video Question Answeringcited by 16
- 2024Long Story Short: Story-level Video Understanding from 20K Short Filmscited by 11
- 2025V-Stylist: Video Stylization via Collaboration and Reflection of MLLM Agentscited by 8
- 2025Any2Caption:Interpreting Any Condition to Caption for Controllable Video Generationcited by 8
Built on by
Citing papers per year: 2026: 15.
15 papers.
Sort:
By area:
- 2026ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creationcited by 7
- 2026Aurora: Unified Video Editing with a Tool-Using Agentcited by 7
- 2026AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks?cited by 7
- 2026UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gatingcited by 4
- 2026DramaDirector: Geometry-Guided Short Drama Generationcited by 2
- 2026Better Call CineCrew: Consistent Ultra-Long Narrative-to-Film Generationcited by 2
- 2026Plans You Can Check: Verifier-Grounded Learning of an Open-Weight Planner for Executable Video-Editingcited by 1
- 2026TOBench: A Task-Oriented Omni-Modal Benchmark for Real-World Tool-Using Agentscited by 0
- 2026Thinking on Shots: Consistent Multi-Shot Video Editing with Agentic Reasoningcited by 0
- 2026FRAMEWORKERS: A Dynamic Multi-Agent Framework for AI-Generated Video Productioncited by 0
- 2026DramaChain Bench: An End-to-End Benchmark for Short-Drama Generationcited by 0
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026VAMR: Multi-Question Agentic Reasoning for Efficient Long-Form Video Understanding8 Oct 2026
- 2026TemplateCraft: Agentic Visual Template Generation25 Sep 2026
- 2026MVAgent: Multi-Agent Video Generation via Consistent Condition Construction and Shot-Level Policy Optimization24 Sep 2026
- 2026Better Call CineCrew: Consistent Ultra-Long Narrative-to-Film Generation7 Sep 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026DramaChain Bench: An End-to-End Benchmark for Short-Drama Generation1 Sep 2026
- 2026FRAMEWORKERS: A Dynamic Multi-Agent Framework for AI-Generated Video Production30 Aug 2026
- 2026Thinking on Shots: Consistent Multi-Shot Video Editing with Agentic Reasoning27 Aug 2026
- 2026Plans You Can Check: Verifier-Grounded Learning of an Open-Weight Planner for Executable Video-Editing26 Aug 2026
- 2026DramaDirector: Geometry-Guided Short Drama Generation23 Jun 2026
- 2026UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating19 Jun 2026
- 2026AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks?26 May 2026
- 2026AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks?cited by 7
- 2026FRAMEWORKERS: A Dynamic Multi-Agent Framework for AI-Generated Video Productioncited by 0
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026MVAgent: Multi-Agent Video Generation via Consistent Condition Construction and Shot-Level Policy Optimizationcited by 0
- 2026TemplateCraft: Agentic Visual Template Generationcited by 0
- 2026TemplateCraft: Agentic Visual Template Generation25 Sep 2026
- 2026MVAgent: Multi-Agent Video Generation via Consistent Condition Construction and Shot-Level Policy Optimization24 Sep 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026FRAMEWORKERS: A Dynamic Multi-Agent Framework for AI-Generated Video Production30 Aug 2026
- 2026AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks?26 May 2026
- 2026Aurora: Unified Video Editing with a Tool-Using Agentcited by 7
- 2026Aurora: Unified Video Editing with a Tool-Using Agent18 May 2026
- 2026ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creationcited by 7
- 2026UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gatingcited by 4
- 2026DramaDirector: Geometry-Guided Short Drama Generationcited by 2
- 2026Better Call CineCrew: Consistent Ultra-Long Narrative-to-Film Generationcited by 2
- 2026Plans You Can Check: Verifier-Grounded Learning of an Open-Weight Planner for Executable Video-Editingcited by 1
- 2026TOBench: A Task-Oriented Omni-Modal Benchmark for Real-World Tool-Using Agentscited by 0
- 2026DramaChain Bench: An End-to-End Benchmark for Short-Drama Generationcited by 0
- 2026VAMR: Multi-Question Agentic Reasoning for Efficient Long-Form Video Understandingcited by 0
- 2026VAMR: Multi-Question Agentic Reasoning for Efficient Long-Form Video Understanding8 Oct 2026
- 2026Better Call CineCrew: Consistent Ultra-Long Narrative-to-Film Generation7 Sep 2026
- 2026DramaChain Bench: An End-to-End Benchmark for Short-Drama Generation1 Sep 2026
- 2026Plans You Can Check: Verifier-Grounded Learning of an Open-Weight Planner for Executable Video-Editing26 Aug 2026
- 2026DramaDirector: Geometry-Guided Short Drama Generation23 Jun 2026
- 2026UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating19 Jun 2026
- 2026TOBench: A Task-Oriented Omni-Modal Benchmark for Real-World Tool-Using Agents16 May 2026
- 2026ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation12 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2024OmAgent: A Multi-modal Agent Framework for Complex Video Understanding with Task Divide-and-Conquer
- 2026VideoAgent: All-in-One Framework for Video Understanding and Editing
- 2025UniVid: The Open-Source Unified Video Model
- 2024Mora: Enabling Generalist Video Generation via A Multi-Agent Framework
- 2026Qwen3.8-Omni: Towards Native Omni-Modal Agents
- 2026VideoWeaver: Evaluating and Evolving Skills for Agentic Long Video Generation
- 2026Agentic Video Understanding: A Survey
- 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaboration