mlsanityData to 7 Oct 2026

UniVA: Universal Video Agent towards Open-Source Next-Generation Video Generalist

arXiv 2511.08521 ↗First posted 11 Nov 2025cs.CVcited by 15

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

While specialized AI models excel at isolated video tasks like generation or understanding, real-world applications demand complex, iterative workflows that combine these capabilities. To bridge this gap, we introduce UniVA, an open-source, omni-capable multi-agent framework for next-generation video generalists that unifies video understanding, segmentation, editing, and generation into cohesive workflows. UniVA employs a Plan-and-Act dual-agent architecture that drives a highly automated and proactive workflow: a planner agent interprets user intentions and decomposes them into structured video-processing steps, while executor agents execute these through modular, MCP-based tool servers (for analysis, generation, editing, tracking, etc.). Through a hierarchical multi-level memory (global knowledge, task context, and user-specific preferences), UniVA sustains long-horizon reasoning, contextual continuity, and inter-agent communication, enabling interactive and self-reflective video creation with full traceability. This design enables iterative and any-conditioned video workflows (e.g., text/image/video-conditioned generation rightarrow multi-round editing rightarrow object segmentation rightarrow compositional synthesis) that were previously cumbersome to achieve with single-purpose models or monolithic video-language models. We also introduce UniVA-Bench, a benchmark suite of multi-step video tasks spanning understanding, editing, segmentation, and generation, to rigorously evaluate such agentic video systems. Both UniVA and UniVA-Bench are fully open-sourced, aiming to catalyze research on interactive, agentic, and general-purpose video intelligence for the next generation of multimodal AI systems. (https://univa.online/)

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 53 of its 60 references that are in the map. First posted on arXiv 11 Nov 2025.

Builds on

53 of 60 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025PreMind: Multi-Agent Video Understanding for Advanced Indexing of Presentation-style Videos28 Feb
  2. 2024Enhancing Video-LLM Reasoning via Agent-of-Thoughts Distillation2 Dec
  3. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding18 Mar
  4. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agent15 Mar
  5. 2024LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing15 Feb

Other references · 48

  1. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  3. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  4. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  5. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  6. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  7. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  8. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  10. 2023A Survey on Multimodal Large Language Modelscited by 898
  11. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  12. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
36 more
  1. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  2. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  3. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  4. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  5. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
  6. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  7. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
  8. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  9. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  10. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
  11. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  12. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 200
  13. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  14. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  15. 2020Hierarchical Conditional Relation Networks for Video Question Answeringcited by 163
  16. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 154
  17. 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 153
  18. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 151
  19. 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
  20. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  21. 2022Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?cited by 90
  22. 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 84
  23. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  24. 2024Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editingcited by 49
  25. 2025MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encodercited by 44
  26. 2022Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Groundingcited by 40
  27. 2025Omni-Video: Democratizing Unified Video Understanding and Generationcited by 39
  28. 2025ThinkSound: Chain-of-Thought Reasoning in Multimodal Large Language Models for Audio Generation and Editingcited by 33
  29. 2024StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaborationcited by 28
  30. 2024VideoAgent: Self-Improving Video Generationcited by 24
  31. 2014A Discriminative CNN Video Representation for Event Detectioncited by 20
  32. 2025On Path to Multimodal Generalist: General-Level and General-Benchcited by 18
  33. 2025VideoMultiAgents: A Multi-Agent Framework for Video Question Answeringcited by 16
  34. 2024Long Story Short: Story-level Video Understanding from 20K Short Filmscited by 11
  35. 2025V-Stylist: Video Stylization via Collaboration and Reflection of MLLM Agentscited by 8
  36. 2025Any2Caption:Interpreting Any Condition to Caption for Controllable Video Generationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 15.

15 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2024OmAgent: A Multi-modal Agent Framework for Complex Video Understanding with Task Divide-and-Conquer
  2. 2026VideoAgent: All-in-One Framework for Video Understanding and Editing
  3. 2025UniVid: The Open-Source Unified Video Model
  4. 2024Mora: Enabling Generalist Video Generation via A Multi-Agent Framework
  5. 2026Qwen3.8-Omni: Towards Native Omni-Modal Agents
  6. 2026VideoWeaver: Evaluating and Evolving Skills for Agentic Long Video Generation
  7. 2026Agentic Video Understanding: A Survey
  8. 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaboration