# MotionCrafter: One-Shot Motion Customization of Diffusion Models

arXiv: https://arxiv.org/abs/2312.05288 · HTML: https://mlsanity.com/paper/2312.05288/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 28 of its 30 references that are in the map.
- First posted on arXiv 8 Dec 2023.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 28 of 28)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2212.11565/) (2022)
- [Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators](https://mlsanity.com/paper/2303.13439/) (2023)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [VideoCrafter1: Open Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2310.19512/) (2023)
- [VideoComposer: Compositional Video Synthesis with Motion Controllability](https://mlsanity.com/paper/2306.02018/) (2023)
- [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023)
- [Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery](https://mlsanity.com/paper/2302.03668/) (2023)
- [VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2303.08320/) (2023)
- [ControlVideo: Training-free Controllable Text-to-Video Generation](https://mlsanity.com/paper/2305.13077/) (2023)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation](https://mlsanity.com/paper/2306.07954/) (2023)
- [P+: Extended Textual Conditioning in Text-to-Image Generation](https://mlsanity.com/paper/2303.09522/) (2023)
- [Break-A-Scene: Extracting Multiple Concepts from a Single Image](https://mlsanity.com/paper/2305.16311/) (2023)
- [Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation](https://mlsanity.com/paper/2305.10874/) (2023)
- [Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation](https://mlsanity.com/paper/2307.06940/) (2023)
- [ReVersion: Diffusion-Based Relation Inversion from Images](https://mlsanity.com/paper/2303.13495/) (2023)
- [ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models](https://mlsanity.com/paper/2305.16225/) (2023)
- [TaleCrafter: Interactive Story Visualization with Multiple Characters](https://mlsanity.com/paper/2305.18247/) (2023)

## Built on by (most cited first, 20 of 20)

- [Controllable Generation with Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2403.04279/) (2024)
- [Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuning](https://mlsanity.com/paper/2506.05207/) (2025)
- [Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models](https://mlsanity.com/paper/2402.14780/) (2024)
- [UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editing](https://mlsanity.com/paper/2402.13185/) (2024)
- [A Survey on Personalized Content Synthesis with Diffusion Models](https://mlsanity.com/paper/2405.05538/) (2024)
- [DreamMotion: Space-Time Self-Similar Score Distillation for Zero-Shot Video Editing](https://mlsanity.com/paper/2403.12002/) (2024)
- [FastVMT: Eliminating Redundancy in Video Motion Transfer](https://mlsanity.com/paper/2602.05551/) (2026)
- [Dynamic Concepts Personalization from Single Videos](https://mlsanity.com/paper/2502.14844/) (2025)
- [MotionFlow: Attention-Driven Motion Transfer in Video Diffusion Models](https://mlsanity.com/paper/2412.05275/) (2024)
- [VFX Creator: Animated Visual Effect Generation with Controllable Diffusion Transformer](https://mlsanity.com/paper/2502.05979/) (2025)
- [SketchVideo: Sketch-based Video Generation and Editing](https://mlsanity.com/paper/2503.23284/) (2025)
- [MotionShop: Zero-Shot Motion Transfer in Video Diffusion Models with Mixture of Score Guidance](https://mlsanity.com/paper/2412.05355/) (2024)
- [SynMotion: Semantic-Visual Adaptation for Motion Customized Video Generation](https://mlsanity.com/paper/2506.23690/) (2025)
- [DreamRunner: Fine-Grained Compositional Story-to-Video Generation with Retrieval-Augmented Motion Adaptation](https://mlsanity.com/paper/2411.16657/) (2024)
- [Reenact Anything: Semantic Video Motion Transfer Using Motion-Textual Inversion](https://mlsanity.com/paper/2408.00458/) (2024)
- [MultiMotion: Multi Subject Video Motion Transfer via Video Diffusion Transformer](https://mlsanity.com/paper/2512.07500/) (2025)
- [Searching Priors Makes Text-to-Video Synthesis Better](https://mlsanity.com/paper/2406.03215/) (2024)
- [Learning to Animate Images from A Few Videos to Portray Delicate Human Actions](https://mlsanity.com/paper/2503.00276/) (2025)
- [Zero-Shot Personalized Camera Motion Control for Image-to-Video Synthesis](https://mlsanity.com/paper/2504.09472/) (2025)
- [Diverse Motion Customization via Control-based Dynamic Optimization](https://mlsanity.com/paper/2610.07911/) (2026)

## Abstract (arXiv)

The essence of a video lies in its dynamic motions, including character actions, object movements, and camera movements. While text-to-video generative diffusion models have recently advanced in creating diverse contents, controlling specific motions through text prompts remains a significant challenge. A primary issue is the coupling of appearance and motion, often leading to overfitting on appearance. To tackle this challenge, we introduce MotionCrafter, a novel one-shot instance-guided motion customization method. MotionCrafter employs a parallel spatial-temporal architecture that injects the reference motion into the temporal component of the base model, while the spatial module is independently adjusted for character or style control. To enhance the disentanglement of motion and appearance, we propose an innovative dual-branch motion disentanglement approach, comprising a motion disentanglement loss and an appearance prior enhancement strategy. During training, a frozen base model provides appearance normalization, effectively separating appearance from motion and thereby preserving diversity. Comprehensive quantitative and qualitative experiments, along with user preference tests, demonstrate that MotionCrafter can successfully integrate dynamic motions while preserving the coherence and quality of the base model with a wide range of appearance generation capabilities. Project page: https://zyxelsa.github.io/homepage-motioncrafter. Codes are available at https://github.com/zyxElsa/MotionCrafter.