# DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Control

arXiv: https://arxiv.org/abs/2410.13830 · HTML: https://mlsanity.com/paper/2410.13830/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 93 of its 101 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 30 of 93)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https://mlsanity.com/paper/2301.07093/) (2023)
- [VILA: On Pre-training for Visual Language Models](https://mlsanity.com/paper/2312.07533/) (2023)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)

## Built on by (most cited first, 30 of 43)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model](https://mlsanity.com/paper/2411.19108/) (2024)
- [MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidance](https://mlsanity.com/paper/2503.16421/) (2025)
- [VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Control](https://mlsanity.com/paper/2501.01427/) (2025)
- [MagCache: Fast Video Generation with Magnitude-Aware Cache](https://mlsanity.com/paper/2506.09045/) (2025)
- [MultiShotMaster: A Controllable Multi-Shot Video Generation Framework](https://mlsanity.com/paper/2512.03041/) (2025)
- [Reangle-A-Video: 4D Video Generation as Video-to-Video Translation](https://mlsanity.com/paper/2503.09151/) (2025)
- [Exploring the Evolution of Physics Cognition in Video Generation: A Survey](https://mlsanity.com/paper/2503.21765/) (2025)
- [PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation](https://mlsanity.com/paper/2411.17048/) (2024)
- [ContextFlow: Training-Free Video Object Editing via Adaptive Context Enrichment](https://mlsanity.com/paper/2509.17818/) (2025)
- [Routing Matters in MoE: Scaling Diffusion Transformers with Explicit Routing Guidance](https://mlsanity.com/paper/2510.24711/) (2025)
- [VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generation](https://mlsanity.com/paper/2502.12782/) (2025)
- [DreamRelation: Relation-Centric Video Customization](https://mlsanity.com/paper/2503.07602/) (2025)
- [MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglement](https://mlsanity.com/paper/2505.23742/) (2025)
- [CINEMA: Coherent Multi-Subject Video Generation via MLLM-Based Guidance](https://mlsanity.com/paper/2503.10391/) (2025)
- [Hybrid-Level Instruction Injection for Video Token Compression in Multi-modal Large Language Models](https://mlsanity.com/paper/2503.16036/) (2025)
- [MagicID: Hybrid Preference Optimization for ID-Consistent and Dynamic-Preserved Video Customization](https://mlsanity.com/paper/2503.12689/) (2025)
- [OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditions](https://mlsanity.com/paper/2506.23361/) (2025)
- [Mimir: Improving Video Diffusion Models for Precise Text Understanding](https://mlsanity.com/paper/2412.03085/) (2024)
- [Survey of Video Diffusion Models: Foundations, Implementations, and Applications](https://mlsanity.com/paper/2504.16081/) (2025)
- [CustomVideoX: 3D Reference Attention Driven Dynamic Adaptation for Zero-Shot Customized Video Diffusion Transformers](https://mlsanity.com/paper/2502.06527/) (2025)
- [Animate-X++: Universal Character Image Animation with Dynamic Backgrounds](https://mlsanity.com/paper/2508.09454/) (2025)
- [MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation](https://mlsanity.com/paper/2605.20183/) (2026)
- [Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory](https://mlsanity.com/paper/2608.29910/) (2026)
- [DreamRunner: Fine-Grained Compositional Story-to-Video Generation with Retrieval-Augmented Motion Adaptation](https://mlsanity.com/paper/2411.16657/) (2024)
- [Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learning](https://mlsanity.com/paper/2512.09924/) (2025)
- [LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation](https://mlsanity.com/paper/2604.11789/) (2026)
- [ShowTable: Unlocking Creative Table Visualization with Collaborative Reflection and Refinement](https://mlsanity.com/paper/2512.13303/) (2025)
- [FlashMotion: Few-Step Controllable Video Generation with Trajectory Guidance](https://mlsanity.com/paper/2603.12146/) (2026)

## Abstract (arXiv)

Recent advances in customized video generation have enabled users to create videos tailored to both specific subjects and motion trajectories. However, existing methods often require complicated test-time fine-tuning and struggle with balancing subject learning and motion control, limiting their real-world applications. In this paper, we present DreamVideo-2, a zero-shot video customization framework capable of generating videos with a specific subject and motion trajectory, guided by a single image and a bounding box sequence, respectively, and without the need for test-time fine-tuning. Specifically, we introduce reference attention, which leverages the model's inherent capabilities for subject learning, and devise a mask-guided motion module to achieve precise motion control by fully utilizing the robust motion signal of box masks derived from bounding boxes. While these two components achieve their intended functions, we empirically observe that motion control tends to dominate over subject learning. To address this, we propose two key designs: 1) the masked reference attention, which integrates a blended latent mask modeling scheme into reference attention to enhance subject representations at the desired positions, and 2) a reweighted diffusion loss, which differentiates the contributions of regions inside and outside the bounding boxes to ensure a balance between subject and motion control. Extensive experimental results on a newly curated dataset demonstrate that DreamVideo-2 outperforms state-of-the-art methods in both subject customization and motion control. The dataset, code, and models will be made publicly available.