mlsanityData to 7 Oct 2026

UniVideo: Unified Understanding, Generation, and Editing for Videos

arXiv 2510.08377 ↗First posted 9 Oct 2025cs.CVcited by 85

Not on the map. Why?

Abstract (arXiv)

Unified multimodal models have shown promising results in multimodal content generation and editing but remain largely limited to the image domain. In this work, we present UniVideo, a versatile framework that extends unified modeling to the video domain. UniVideo adopts a dual-stream design, combining a Multimodal Large Language Model (MLLM) for instruction understanding with a Multimodal DiT (MMDiT) for video generation. This design preserves the MLLM's original text generation capabilities, enables accurate interpretation of complex multimodal instructions, and maintains visual consistency in the generated content. Built on this architecture, UniVideo unifies diverse video generation and editing tasks under a single multimodal instruction paradigm and is jointly trained across them. Extensive experiments demonstrate that UniVideo matches or surpasses state-of-the-art task-specific baselines in text/image-to-video generation, in-context video generation and in-context video editing. Notably, the unified design of UniVideo enables two forms of generalization. First, UniVideo supports task composition, such as combining editing with style transfer, by integrating multiple capabilities within a single instruction. Second, even without explicit training on free-form video editing, UniVideo transfers its editing capability from large-scale image editing data to this setting, handling unseen instructions such as changing the environment or altering materials within a video. Beyond these core capabilities, UniVideo also supports visual-prompt-based video generation, where the MLLM interprets visual prompts and guides the MMDiT during synthesis. To foster future research, we released our model and code.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 81 of its 84 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

81 of 84 references.

References · 81

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  12. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
38 more
  1. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  3. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  7. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  8. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  9. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  10. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  11. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  12. 2025Qwen-Image Technical Reportcited by 973
  13. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  14. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  15. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  16. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  17. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  18. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  19. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  20. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  21. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  22. 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
  23. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  24. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  25. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  26. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  27. 2025VACE: All-in-One Video Creation and Editingcited by 467
  28. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  29. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  30. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  31. 2024OmniGen: Unified Image Generationcited by 388
  32. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  33. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
  34. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  35. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  36. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
  37. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  38. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 70.

85 papers.

Sort:

By area:

  1. 2025Scaling Instruction-Based Video Editing with a High-Quality Synthetic Datasetcited by 76
  2. 2025OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editingcited by 50
  3. 2026Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidancecited by 34
  4. 2026Beyond Language Modeling: An Exploration of Multimodal Pretrainingcited by 32
  5. 2025IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessmentcited by 27
  6. 2026OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoningcited by 25
  7. 2026Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editingcited by 18
  8. 2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 17
  9. 2025VIVA: VLM-Guided Instruction-Based Video Editing with Reward Optimizationcited by 16
  10. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  11. 2025JavisGPT: A Unified Multi-modal LLM for Sounding-Video Comprehension and Generationcited by 12
  12. 2026Lance: Unified Multimodal Modeling by Multi-Task Synergycited by 11
38 more
  1. 2025Visual-Aware CoT: Achieving High-Fidelity Visual Consistency in Unified Modelscited by 10
  2. 2026LLaMo: Scaling Pretrained Language Models for Unified Motion Understanding and Generation with Continuous Autoregressive Tokenscited by 10
  3. 2026Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editingcited by 10
  4. 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forwardcited by 9
  5. 2026PISCO: Precise Video Instance Insertion with Sparse Controlcited by 9
  6. 2026Tele-Omni: a Unified Multimodal Framework for Video Generation and Editingcited by 8
  7. 2026SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editingcited by 8
  8. 2025Video-as-Answer: Predict and Generate Next Video Event with Joint-GRPOcited by 7
  9. 2026LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Modelcited by 7
  10. 2026Aurora: Unified Video Editing with a Tool-Using Agentcited by 7
  11. 2025iMontage: Unified, Versatile, Highly Dynamic Many-to-many Image Generationcited by 6
  12. 2026PhyRPR: Training-Free Physics-Constrained Video Generationcited by 6
  13. 2026A Mechanistic View on Video Generation as World Models: State and Dynamicscited by 6
  14. 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editingcited by 6
  15. 2025VideoCoF: Unified Video Editing with Temporal Reasonercited by 5
  16. 2026UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priorscited by 5
  17. 2026Video Understanding: From Geometry and Semantics to Unified Modelscited by 5
  18. 2026VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editingcited by 5
  19. 2025MAViD: A Multimodal Framework for Audio-Visual Dialogue Understanding and Generationcited by 4
  20. 2025Exploring MLLM-Diffusion Information Transfer with MetaCanvascited by 4
  21. 2026AutoCut: End-to-end advertisement video editing based on multimodal discretization and controllable generationcited by 4
  22. 2026VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effectscited by 4
  23. 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusioncited by 4
  24. 2026Gloria: Consistent Character Video Generation via Content Anchorscited by 3
  25. 2026Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoEcited by 3
  26. 2026LIVEditor-14B: Lightning Unified Video Editing via In-Context Sparse Attentioncited by 3
  27. 2026Smart-Insertion-V: Photorealistic Video Insertion via a Closed-Loop Feedback Dual-Stream Frameworkcited by 3
  28. 2026SteerVTE: Seamless Video Text Editing with Style and Glyph Controlcited by 3
  29. 2025What Happens Next? Next Scene Prediction with a Unified Video Modelcited by 2
  30. 2026Physics-Aware Video Instance Removal Benchmarkcited by 2
  31. 2026How Far Are Video Models from True Multimodal Reasoning?cited by 2
  32. 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generationcited by 2
  33. 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationshipscited by 2
  34. 2026Instruction-Based Video Editing by Repurposing an Image Editing Modelcited by 2
  35. 2026GRNEdit: Efficient General Video Editing from a New Binary-Evidence Perspective in Generative Refinement Networkscited by 2
  36. 2026VGI-Bench: Probing Visual Intelligence in Video Generation Modelscited by 2
  37. 2026InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptationcited by 1
  38. 2026LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editingcited by 1

50 most cited of 85 All 85 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026UniVBench: Towards Unified Evaluation for Video Foundation Models
  2. 2026VINO: A Unified Visual Generator with Interleaved OmniModal Context
  3. 2026Uni-ViGU: Towards Unified Video Generation and Understanding via A Diffusion-Based Video Generator
  4. 2025MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing
  5. 2025InstructX: Towards Unified Visual Editing with MLLM Guidance
  6. 2026DuoGen: Towards General Purpose Interleaved Multimodal Generation
  7. 2025UniVG: A Generalist Diffusion Model for Unified Image Generation and Editing
  8. 2025Omni-Video: Democratizing Unified Video Understanding and Generation