DreamVideo-Omni: Omni-Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learning
On the map: Diffusion models → Video generation
Abstract (arXiv)
While large-scale diffusion models have revolutionized video synthesis, achieving precise control over both multi-subject identity and multi-granularity motion remains a significant challenge. Recent attempts to bridge this gap often suffer from limited motion granularity, control ambiguity, and identity degradation, leading to suboptimal performance on identity preservation and motion control. In this work, we present DreamVideo-Omni, a unified framework enabling harmonious multi-subject customization with omni-motion control via a progressive two-stage training paradigm. In the first stage, we integrate comprehensive control signals for joint training, encompassing subject appearances, global motion, local dynamics, and camera movements. To ensure robust and precise controllability, we introduce a condition-aware 3D rotary positional embedding to coordinate heterogeneous inputs and a hierarchical motion injection strategy to enhance global motion guidance. Furthermore, to resolve multi-subject ambiguity, we introduce group and role embeddings to explicitly anchor motion signals to specific identities, effectively disentangling complex scenes into independent controllable instances. In the second stage, to mitigate identity degradation, we design a latent identity reward feedback learning paradigm by training a latent identity reward model upon a pretrained video diffusion backbone. This provides motion-aware identity rewards in the latent space, prioritizing identity preservation aligned with human preferences. Supported by our curated large-scale dataset and the comprehensive DreamOmni Bench for multi-subject and omni-motion control evaluation, DreamVideo-Omni demonstrates superior performance in generating high-quality videos with precise controllability.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 98 of its 106 references that are in the map. First posted on arXiv 12 Mar 2026.
Builds on
98 of 106 references.
Closest ancestors · 33
Same direction, newest first
- 2025Identity-Preserving Image-to-Video Generation via Reward-Guided Optimization16 Oct
- 2025Tora2: Motion and Appearance Customized Diffusion Transformer for Multi-Entity Video Generation8 Jul
- 2025VideoMage: Multi-Subject and Motion Customization of Text-to-Video Diffusion Models27 Mar
- 2025VACE: All-in-One Video Creation and Editing10 Mar
- 2025CustomVideoX: 3D Reference Attention Driven Dynamic Adaptation for Zero-Shot Customized Video Diffusion Transformers10 Feb
- 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generation6 Feb
- 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuning8 Jan
- 2024VideoMaker: Zero-shot Customized Video Generation with the Inherent Force of Video Diffusion Models27 Dec
- 2024MotionShop: Zero-Shot Motion Transfer in Video Diffusion Models with Mixture of Score Guidance6 Dec
- 2024DIVE: Taming DINO for Subject-Driven Video Editing4 Dec
- 2024MoTrans: Customized Motion Transfer with Text-driven Video Diffusion Models2 Dec
- 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Model21 Oct
21 more
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generation31 Jul
- 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model30 May
- 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation2 May
- 2024Spectral Motion Alignment for Video Motion Transfer using Diffusion Models22 Mar
- 2024DreamMotion: Space-Time Self-Similar Score Distillation for Zero-Shot Video Editing18 Mar
- 2024Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models22 Feb
- 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion5 Feb
- 2024Motion-Zero: Zero-Shot Moving Object Control Framework for Diffusion-Based Video Generation18 Jan
- 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjects18 Jan
- 2023TrailBlazer: Trajectory Control for Diffusion-Based Video Generation31 Dec
- 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motion7 Dec
- 2023VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models1 Dec
- 2023VideoBooth: Diffusion-based Video Generation with Image Prompts1 Dec
- 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generation30 Oct
- 2023LAMP: Learn A Motion Pattern for Few-Shot-Based Video Generation16 Oct
- 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models26 Sep
- 2023ModelScope Text-to-Video Technical Report12 Aug
- 2023Dreamix: Video Diffusion Models are General Video Editors2 Feb
- 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov
- 2022Video Diffusion Models7 Apr
Other references · 65
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
38 more
- 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
- 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 471
- 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 441
- 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
- 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
- 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
- 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
- 2023Recognize Anything: A Strong Image Tagging Modelcited by 305
- 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
- 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
- 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
- 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
- 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
- 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
- 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
- 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 126
- 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
- 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 122
- 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
- 2024MotionBooth: Motion-Aware Customized Text-to-Video Generationcited by 92
- 2023PEEKABOO: Interactive Video Generation via Masked-Diffusioncited by 76
- 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
- 2023DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generationcited by 75
- 2024SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generationcited by 68
- 2024Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Controlcited by 66
- 2025Multi-subject Open-set Personalization in Video Generationcited by 65
- 2024Image Conductor: Precision Control for Interactive Video Synthesiscited by 62
- 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 60
- 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 58
- 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilitiescited by 58
- 2024Still-Moving: Customized Video Generation without Customized Video Datacited by 49
- 2025FullDiT: Multi-Task Video Generative Foundation Model with Full Attentioncited by 45
- 2025Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidancecited by 45
- 2024DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Controlcited by 42
50 most cited of 65 All 65 (JSON)
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generationcited by 8
- 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationshipscited by 2
- 2026Bridging Brain and Semantics: A Hierarchical Framework for Semantically Enhanced fMRI-to-Video Reconstructioncited by 0
- 2026ContextMaster: Interactive Multi-Shot Video Creation via Fixed-Budget Sparse Context Routingcited by 0
- 2026ContextMaster: Interactive Multi-Shot Video Creation via Fixed-Budget Sparse Context Routing5 Aug 2026
- 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationships16 Jul 2026
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation19 May 2026
- 2026Bridging Brain and Semantics: A Hierarchical Framework for Semantically Enhanced fMRI-to-Video Reconstruction14 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025DualReal: Adaptive Joint Training for Lossless Identity-Motion Fusion in Video Customization
- 2023DreaMoving: A Human Video Generation Framework based on Diffusion Models
- 2025Proteus-ID: ID-Consistent and Motion-Coherent Video Customization
- 2026Tri-Prompting: Video Diffusion with Unified Control over Scene, Subject, and Motion
- 2025PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancement
- 2026DreamActor-M2: Universal Character Image Animation via Spatiotemporal In-Context Learning
- 2025Stable Video-Driven Portraits
- 2026AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation