mlsanityData to 7 Oct 2026

Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models

arXiv 2402.14780 ↗First posted 22 Feb 2024cs.CVcited by 55

On the map: Diffusion models → Video generation

Abstract (arXiv)

Image customization has been extensively studied in text-to-image (T2I) diffusion models, leading to impressive outcomes and applications. With the emergence of text-to-video (T2V) diffusion models, its temporal counterpart, motion customization, has not yet been well investigated. To address the challenge of one-shot video motion customization, we propose Customize-A-Video that models the motion from a single reference video and adapts it to new subjects and scenes with both spatial and temporal varieties. It leverages low-rank adaptation (LoRA) on temporal attention layers to tailor the pre-trained T2V diffusion model for specific motion modeling. To disentangle the spatial and temporal information during training, we introduce a novel concept of appearance absorbers that detach the original appearance from the reference video prior to motion learning. The proposed modules are trained in a staged pipeline and inferred in a plug-and-play fashion, enabling easy extensions to various downstream tasks such as custom video generation and editing, video appearance customization and multiple motion combination. Our project page can be found at https://customize-a-video.github.io.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 56 of its 64 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

56 of 64 references.

Closest ancestors · 27

Same direction, newest first

  1. 2023MotionCrafter: One-Shot Motion Customization of Diffusion Models8 Dec
  2. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motion7 Dec
  3. 2023VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models1 Dec
  4. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generation30 Oct
  5. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Models12 Oct
  6. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editing18 Aug
  7. 2023Dual-Stream Diffusion Net for Text-to-Video Generation16 Aug
  8. 2023ModelScope Text-to-Video Technical Report12 Aug
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  10. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation13 Jun
  11. 2023Video ControlNet: Towards Temporally Consistent Synthetic-to-Real Video Translation Using Conditional Image Diffusion Models30 May
  12. 2023Towards Consistent Video Editing with Text-to-Image Diffusion Models27 May
15 more
  1. 2023ControlVideo: Training-free Controllable Text-to-Video Generation22 May
  2. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models17 May
  3. 2023Make-A-Protagonist: Generic Video Editing with An Ensemble of Experts15 May
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  5. 2023Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Models30 Mar
  6. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
  7. 2023Pix2Video: Video Editing using Image Diffusion22 Mar
  8. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editing16 Mar
  9. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation15 Mar
  10. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  11. 2023Dreamix: Video Diffusion Models are General Video Editors2 Feb
  12. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
  13. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov
  14. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  15. 2022Video Diffusion Models7 Apr

Other references · 29

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  7. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  8. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  9. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  10. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  11. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  12. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
17 more
  1. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  2. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  3. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  4. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  5. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  6. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  7. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  8. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
  9. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  10. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  11. 2017Video Generation From Textcited by 153
  12. 2018To Create What You Tell: Generating Videos from Captionscited by 84
  13. 2023MagicEdit: High-Fidelity and Temporally Coherent Video Editingcited by 75
  14. 2023Edit-A-Video: Single Video Editing with Object-Aware Consistencycited by 65
  15. 2023CVPR 2023 Text Guided Video Editing Competitioncited by 54
  16. 2020TiVGAN: Text to Image to Video Generation with Step-by-Step Evolutionary Generatorcited by 18
  17. 2023NewMove: Customizing text-to-video models with novel motionscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 19; 2025: 27; 2026: 9.

55 papers.

Sort:

By area:

  1. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  2. 2025Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuningcited by 81
  3. 2024A Survey on LoRA of Large Language Modelscited by 67
  4. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  5. 2024DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Controlcited by 42
  6. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  7. 2024Customizing Text-to-Image Models with a Single Image Paircited by 36
  8. 2024Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generationcited by 32
  9. 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
  10. 2024Videoshop: Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversioncited by 27
  11. 2024SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Devicecited by 25
  12. 2025Human Motion Video Generation: A Surveycited by 25
38 more
  1. 2025Generative Physical AI in Vision: A Surveycited by 24
  2. 2025DreamRelation: Relation-Centric Video Customizationcited by 23
  3. 2026FastVMT: Eliminating Redundancy in Video Motion Transfercited by 23
  4. 2024Spectral Motion Alignment for Video Motion Transfer using Diffusion Modelscited by 20
  5. 2025VideoMage: Multi-Subject and Motion Customization of Text-to-Video Diffusion Modelscited by 17
  6. 2025Generative AI for Film Creation: A Survey of Recent Advancescited by 17
  7. 2024MotionFlow: Attention-Driven Motion Transfer in Video Diffusion Modelscited by 16
  8. 2025VFX Creator: Animated Visual Effect Generation with Controllable Diffusion Transformercited by 15
  9. 2025Decouple and Track: Benchmarking and Improving Video Diffusion Transformers for Motion Transfercited by 14
  10. 2024MotionShop: Zero-Shot Motion Transfer in Video Diffusion Models with Mixture of Score Guidancecited by 13
  11. 2025Ingredients: Blending Custom Photos with Video Diffusion Transformerscited by 13
  12. 2025Video-As-Prompt: Unified Semantic Control for Video Generationcited by 13
  13. 2025MagicTryOn: Harnessing Diffusion Transformer for Garment-Preserving Video Virtual Try-oncited by 12
  14. 2024Edit-Your-Motion: Space-Time Diffusion Decoupling Learning for Video Motion Editingcited by 10
  15. 2026LaVR: Scene Latent Conditioned Generative Video Trajectory Re-Rendering using Large 4D Reconstruction Modelscited by 10
  16. 2024CustomTTT: Motion and Appearance Customized Video Generation via Test-Time Trainingcited by 9
  17. 2025DualReal: Adaptive Joint Training for Lossless Identity-Motion Fusion in Video Customizationcited by 9
  18. 2025Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenizationcited by 9
  19. 2025Visual Prompting for One-shot Controllable Video Editing without Inversioncited by 8
  20. 2024DreamRunner: Fine-Grained Compositional Story-to-Video Generation with Retrieval-Augmented Motion Adaptationcited by 7
  21. 2025MotionRAG: Motion Retrieval-Augmented Image-to-Video Generationcited by 7
  22. 2024Reenact Anything: Semantic Video Motion Transfer Using Motion-Textual Inversioncited by 6
  23. 2026Evolution of Video Generative Foundationscited by 6
  24. 2025JointTuner: Appearance-Motion Adaptive Joint Training for Customized Video Generationcited by 5
  25. 2026DreamVideo-Omni: Omni-Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learningcited by 4
  26. 2024Motion by Queries: Identity-Motion Trade-offs in Text-to-Video Generationcited by 3
  27. 2025MotionMatcher: Motion Customization of Text-to-Video Diffusion Models via Motion Feature Matchingcited by 3
  28. 2025In-Context Sync-LoRA for Portrait Video Editingcited by 3
  29. 2024Beyond Generation: Unlocking Universal Editing via Self-Supervised Fine-Tuningcited by 2
  30. 2026Compression as Adaptation: Implicit Visual Representation with Diffusion Foundation Modelscited by 2
  31. 2024Searching Priors Makes Text-to-Video Synthesis Bettercited by 1
  32. 2025Learning to Animate Images from A Few Videos to Portray Delicate Human Actionscited by 1
  33. 2025RDTF: Resource-efficient Dual-mask Training Framework for Multi-frame Animated Sticker Generationcited by 1
  34. 2025Characterizing Motion Encoding in Video Diffusion Timestepscited by 1
  35. 2026FlowMotion: Training-Free Flow Guidance for Video Motion Transfercited by 1
  36. 2025How to Train Your Dragon: Automatic Diffusion-Based Rigging for Characters with Diverse Topologiescited by 0
  37. 2025Zero-Shot Personalized Camera Motion Control for Image-to-Video Synthesiscited by 0
  38. 2025Interactive Video Generation via Domain Adaptationcited by 0

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Separate Motion from Appearance: Customizing Motion via Customizing Text-to-Video Diffusion Models
  2. 2026MotionAdapter: Video Motion Transfer via Content-Aware Attention Customization
  3. 2024MoTrans: Customized Motion Transfer with Text-driven Video Diffusion Models
  4. 2024Motion Inversion for Video Customization
  5. 2023AnimateZero: Video Diffusion Models are Zero-Shot Image Animators
  6. 2024Animate Your Motion: Turning Still Images into Dynamic Videos
  7. 2025LMP: Leveraging Motion Prior in Zero-Shot Video Generation with Diffusion Transformer
  8. 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities