# Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editing

arXiv: https://arxiv.org/abs/2602.08820 · HTML: https://mlsanity.com/paper/2602.08820/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 20 of 20)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [TokenFlow: Consistent Diffusion Features for Consistent Video Editing](https://mlsanity.com/paper/2307.10373/) (2023)
- [BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset](https://mlsanity.com/paper/2505.09568/) (2025)
- [MetaMorph: Multimodal Understanding and Generation via Instruction Tuning](https://mlsanity.com/paper/2412.14164/) (2024)
- [JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2411.07975/) (2024)
- [AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Tasks](https://mlsanity.com/paper/2403.14468/) (2024)
- [Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer](https://mlsanity.com/paper/2311.17009/) (2023)
- [VidToMe: Video Token Merging for Zero-Shot Video Editing](https://mlsanity.com/paper/2312.10656/) (2023)
- [UniVideo: Unified Understanding, Generation, and Editing for Videos](https://mlsanity.com/paper/2510.08377/) (2025)
- [EasyAnimate: High-Performance Video Generation Framework with Hybrid Windows Attention and Reward Backpropagation](https://mlsanity.com/paper/2405.18991/) (2024)
- [Vchitect-2.0: Parallel Transformer for Scaling Up Video Diffusion Models](https://mlsanity.com/paper/2501.08453/) (2025)
- [UNIC: Unified In-Context Video Editing](https://mlsanity.com/paper/2506.04216/) (2025)
- [EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning](https://mlsanity.com/paper/2509.20360/) (2025)
- [VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing](https://mlsanity.com/paper/2502.17258/) (2025)
- [Omni-Video: Democratizing Unified Video Understanding and Generation](https://mlsanity.com/paper/2507.06119/) (2025)
- [UniVid: The Open-Source Unified Video Model](https://mlsanity.com/paper/2509.24200/) (2025)

## Built on by (most cited first, 19 of 19)

- [OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning](https://mlsanity.com/paper/2603.24458/) (2026)
- [LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing](https://mlsanity.com/paper/2606.06042/) (2026)
- [VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editing](https://mlsanity.com/paper/2608.05485/) (2026)
- [Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE](https://mlsanity.com/paper/2605.02641/) (2026)
- [Uni-ViGU: Towards Unified Video Generation and Understanding via A Diffusion-Based Video Generator](https://mlsanity.com/paper/2604.08121/) (2026)
- [TIDE: Task-Isolated Diffusion for Unified Video Editing and Generation](https://mlsanity.com/paper/2606.08260/) (2026)
- [EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editing](https://mlsanity.com/paper/2608.21424/) (2026)
- [Sparkle: Realizing Lively Instruction-Guided Video Background Replacement via Decoupled Guidance](https://mlsanity.com/paper/2605.06535/) (2026)
- [ImVideoEdit: Image-learning Video Editing via 2D Spatial Difference Attention Blocks](https://mlsanity.com/paper/2604.07958/) (2026)
- [UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs](https://mlsanity.com/paper/2604.15871/) (2026)
- [CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognition](https://mlsanity.com/paper/2605.19995/) (2026)
- [Occlusion-Aware Physics-Semantic Keyframe Selection for Robust Video Editing](https://mlsanity.com/paper/2605.23192/) (2026)
- [Reasoning to Align: Implicit Reasoning in Diffusion Transformers for Video Editing](https://mlsanity.com/paper/2605.24674/) (2026)
- [MoRoute: Dynamic Routing for In-Context Multimodal Video Generation](https://mlsanity.com/paper/2607.29545/) (2026)
- [Beyond Text Conditioning: A Systematic Study of MLLM-DiT Fusion for Video Generation](https://mlsanity.com/paper/2608.14043/) (2026)
- [RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing](https://mlsanity.com/paper/2608.26101/) (2026)
- [Reimagine Video Dynamics](https://mlsanity.com/paper/2609.36496/) (2026)
- [ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing](https://mlsanity.com/paper/2610.08779/) (2026)
- [IntactWorld: Joint World Modeling with Intact Features](https://mlsanity.com/paper/2610.11174/) (2026)

## Abstract (arXiv)

We present Omni-Video 2, a scalable and computationally efficient model that connects pretrained multimodal large-language models (MLLMs) with video diffusion models for unified video generation and editing. Our key idea is to exploit the understanding and reasoning capabilities of MLLMs to produce explicit target captions to interpret user instructions. In this way, the rich contextual representations from the understanding model are directly used to guide the generative process, thereby improving performance on complex and compositional editing. Moreover, a lightweight adapter is developed to inject multimodal conditional tokens into pretrained text-to-video diffusion models, allowing maximum reuse of their powerful generative priors in a parameter-efficient manner. Benefiting from these designs, we scale up Omni-Video 2 to a 14B video diffusion model on meticulously curated training data with quality, supporting high quality text-to-video generation and various video editing tasks such as object removal, addition, background change, complex motion editing, etc. We evaluate the performance of Omni-Video 2 on the FiVE benchmark for fine-grained video editing and the VBench benchmark for text-to-video generation. The results demonstrate its superior ability to follow complex compositional instructions in video editing, while also achieving competitive or superior quality in video generation tasks.