OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning
Not on the map. Why?
Abstract (arXiv)
While proprietary systems such as Seedance-2.0 have achieved remarkable success in omni-capable video generation, the academic research community lags far behind: most of its models remain heavily fragmented, and the few existing efforts toward unified video generation still struggle to seamlessly integrate diverse tasks within a single framework. To bridge this gap, we propose OmniWeaving, an omni-level video generation model featuring powerful multimodal composition and reasoning-informed capabilities. By leveraging a massive-scale pretraining dataset that encompasses diverse compositional and reasoning-augmented scenarios, OmniWeaving learns to temporally bind interleaved text, multi-image, and video inputs while acting as an intelligent agent to infer complex user intentions for sophisticated video creation. Furthermore, we introduce IntelligentVBench, the first comprehensive benchmark designed to rigorously assess next-level intelligent unified video generation. Extensive experiments demonstrate that OmniWeaving achieves SoTA performance among open-source academic unified models. The code and model are publicly available. Project Page: https://omniweaving.github.io.
Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 43 of its 52 references that are in the map. First posted on arXiv 25 Mar 2026.
Builds on
43 of 52 references.
References · 43
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2025Qwen-Image Technical Reportcited by 1,044
- 2025SAM 3: Segment Anything with Conceptscited by 906
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
31 more
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
- 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
- 2025VACE: All-in-One Video Creation and Editingcited by 479
- 2025Muon is Scalable for LLM Trainingcited by 428
- 2024OmniGen: Unified Image Generationcited by 394
- 2025Seed1.5-VL Technical Reportcited by 373
- 2025Transfer between Modalities with MetaQueriescited by 236
- 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 168
- 2025HunyuanVideo 1.5 Technical Reportcited by 160
- 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 129
- 2024StableAnimator: High-Quality Identity-Preserving Human Image Animationcited by 88
- 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 87
- 2025Scaling Instruction-Based Video Editing with a High-Quality Synthetic Datasetcited by 79
- 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 73
- 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
- 2025Kling-Omni Technical Reportcited by 71
- 2025InsViE-1M: Effective Instruction-based Video Editing with Elaborate Dataset Constructioncited by 62
- 2023MotionEditor: Editing Video Motion via Content-Aware Diffusioncited by 57
- 2025OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editingcited by 53
- 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generationcited by 44
- 2024Video Editing via Factorized Diffusion Distillationcited by 41
- 2025Omni-Video: Democratizing Unified Video Understanding and Generationcited by 40
- 2025In-Context Learning with Unpaired Clips for Instruction-based Video Editingcited by 35
- 2024Auto-Encoding Morph-Tokens for Multimodal LLMcited by 30
- 2024DreamOmni: Unified Image Generation and Editingcited by 30
- 2025Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokenscited by 28
- 2026VINO: A Unified Visual Generator with Interleaved OmniModal Contextcited by 26
- 2026SkyReels-V3 Technique Reportcited by 20
- 2026Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editingcited by 19
- 2026ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillationcited by 7
- 2026Preference Score Distillation: Leveraging 2D Rewards to Align Text-to-3D Generation with Human Preferencecited by 2
Built on by
Citing papers per year: 2026: 27.
27 papers.
Sort:
By area:
- 2026PhyGround: Benchmarking Physical Reasoning in Generative World Modelscited by 12
- 2026StreamingEffect: Real-Time Human-Centric Video Effect Generationcited by 8
- 2026VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine Systemcited by 8
- 2026PhyWorld: Physics-Faithful World Model for Video Generationcited by 6
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editingcited by 6
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 6
- 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusioncited by 6
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoningcited by 6
- 2026VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editingcited by 5
- 2026VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generationcited by 5
- 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generationcited by 5
- 2026Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoEcited by 3
15 more
- 2026CoVEBench: Can Video Editing Models Handle Complex Instructions?cited by 3
- 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationshipscited by 3
- 2026SpatialFusion: Endowing Unified Image Generation with Intrinsic 3D Geometric Awarenesscited by 2
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generationcited by 2
- 2026PhiZero: A World Model Built Around Physical Languagecited by 2
- 2026CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editingcited by 2
- 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancementcited by 1
- 2026Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Disseminationcited by 1
- 2026Game2World Engine: Unlocking In-the-Wild Gameplay Videos for World Model Trainingcited by 1
- 2026CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognitioncited by 0
- 2026What Semantics Survive the Connector? Diagnosing VLM-to-DiT Alignment in Video Editingcited by 0
- 2026HumanForge: A Human-Centric Deepfake Video Benchmark with Multi-Agent Forgery Rationalescited by 0
- 2026MoRoute: Dynamic Routing for In-Context Multimodal Video Generationcited by 0
- 2026Breaking the Uniformity Trap: Scaling Video Diffusion Model via SplitMoEcited by 0
- 2026Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Trainingcited by 0
- 2026Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training4 Oct 2026
- 2026Breaking the Uniformity Trap: Scaling Video Diffusion Model via SplitMoE29 Sep 2026
- 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generation10 Sep 2026
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning26 Aug 2026
- 2026Game2World Engine: Unlocking In-the-Wild Gameplay Videos for World Model Training25 Aug 2026
- 2026VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation19 Aug 2026
- 2026CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editing18 Aug 2026
- 2026Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Dissemination14 Aug 2026
- 2026VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editing6 Aug 2026
- 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion4 Aug 2026
- 2026MoRoute: Dynamic Routing for In-Context Multimodal Video Generation31 Jul 2026
- 2026PhiZero: A World Model Built Around Physical Language30 Jul 2026
15 more
- 2026VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System29 Jul 2026
- 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement20 Jul 2026
- 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationships16 Jul 2026
- 2026HumanForge: A Human-Centric Deepfake Video Benchmark with Multi-Agent Forgery Rationales9 Jul 2026
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation8 Jun 2026
- 2026CoVEBench: Can Video Editing Models Handle Complex Instructions?7 Jun 2026
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generation6 Jun 2026
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing4 Jun 2026
- 2026What Semantics Survive the Connector? Diagnosing VLM-to-DiT Alignment in Video Editing20 May 2026
- 2026CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognition19 May 2026
- 2026PhyWorld: Physics-Faithful World Model for Video Generation19 May 2026
- 2026StreamingEffect: Real-Time Human-Centric Video Effect Generation16 May 2026
- 2026PhyGround: Benchmarking Physical Reasoning in Generative World Models11 May 2026
- 2026Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE4 May 2026
- 2026SpatialFusion: Endowing Unified Image Generation with Intrinsic 3D Geometric Awareness29 Apr 2026
- 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationshipscited by 3
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generationcited by 2
- 2026HumanForge: A Human-Centric Deepfake Video Benchmark with Multi-Agent Forgery Rationalescited by 0
- 2026Breaking the Uniformity Trap: Scaling Video Diffusion Model via SplitMoEcited by 0
- 2026Breaking the Uniformity Trap: Scaling Video Diffusion Model via SplitMoE29 Sep 2026
- 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationships16 Jul 2026
- 2026HumanForge: A Human-Centric Deepfake Video Benchmark with Multi-Agent Forgery Rationales9 Jul 2026
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generation6 Jun 2026
- 2026PhyGround: Benchmarking Physical Reasoning in Generative World Modelscited by 12
- 2026PhyWorld: Physics-Faithful World Model for Video Generationcited by 6
- 2026PhiZero: A World Model Built Around Physical Languagecited by 2
- 2026Game2World Engine: Unlocking In-the-Wild Gameplay Videos for World Model Trainingcited by 1
- 2026Game2World Engine: Unlocking In-the-Wild Gameplay Videos for World Model Training25 Aug 2026
- 2026PhiZero: A World Model Built Around Physical Language30 Jul 2026
- 2026PhyWorld: Physics-Faithful World Model for Video Generation19 May 2026
- 2026PhyGround: Benchmarking Physical Reasoning in Generative World Models11 May 2026
- 2026StreamingEffect: Real-Time Human-Centric Video Effect Generationcited by 8
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editingcited by 6
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 6
- 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusioncited by 6
- 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generationcited by 5
- 2026CoVEBench: Can Video Editing Models Handle Complex Instructions?cited by 3
- 2026SpatialFusion: Endowing Unified Image Generation with Intrinsic 3D Geometric Awarenesscited by 2
- 2026CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editingcited by 2
- 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancementcited by 1
- 2026Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Disseminationcited by 1
- 2026MoRoute: Dynamic Routing for In-Context Multimodal Video Generationcited by 0
- 2026Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Trainingcited by 0
- 2026Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training4 Oct 2026
- 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generation10 Sep 2026
- 2026CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editing18 Aug 2026
- 2026Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Dissemination14 Aug 2026
- 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion4 Aug 2026
- 2026MoRoute: Dynamic Routing for In-Context Multimodal Video Generation31 Jul 2026
- 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement20 Jul 2026
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation8 Jun 2026
- 2026CoVEBench: Can Video Editing Models Handle Complex Instructions?7 Jun 2026
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing4 Jun 2026
- 2026StreamingEffect: Real-Time Human-Centric Video Effect Generation16 May 2026
- 2026SpatialFusion: Endowing Unified Image Generation with Intrinsic 3D Geometric Awareness29 Apr 2026