mlsanityData to 7 Oct 2026

MotionClone: Training-Free Motion Cloning for Controllable Video Generation

arXiv 2406.05338 ↗First posted 8 Jun 2024cs.CVcited by 104

Not on the map. Why?

Cited by 44 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Motion-based controllable video generation offers the potential for creating captivating visual content. Existing methods typically necessitate model training to encode particular motion cues or incorporate fine-tuning to inject certain motion patterns, resulting in limited flexibility and generalization. In this work, we propose MotionClone, a training-free framework that enables motion cloning from reference videos to versatile motion-controlled video generation, including text-to-video and image-to-video. Based on the observation that the dominant components in temporal-attention maps drive motion synthesis, while the rest mainly capture noisy or very subtle motions, MotionClone utilizes sparse temporal attention weights as motion representations for motion guidance, facilitating diverse motion transfer across varying scenarios. Meanwhile, MotionClone allows for the direct extraction of motion representation through a single denoising step, bypassing the cumbersome inversion processes and thus promoting both efficiency and flexibility. Extensive experiments demonstrate that MotionClone exhibits proficiency in both global camera motion and local object motion, with notable superiority in terms of motion fidelity, textual alignment, and temporal consistency.

Key facts. Cited by 104 papers in the map (data through 7 Oct 2026). Builds on 42 of its 44 references that are in the map. First posted on arXiv 8 Jun 2024.

Builds on

42 of 44 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  11. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  12. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
30 more
  1. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  2. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  3. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  4. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  5. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  6. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  7. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  8. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  9. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  10. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  11. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  12. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  13. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  14. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  15. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  16. 2024Long-CLIP: Unlocking the Long-Text Capability of CLIPcited by 298
  17. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
  18. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
  19. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wildcited by 197
  20. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuningcited by 191
  21. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Modelscited by 172
  22. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 172
  23. 2023Dense Text-to-Image Generation with Attention Modulationcited by 162
  24. 2023Make-Your-Video: Customized Video Generation Using Textual and Structural Guidancecited by 129
  25. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 89
  26. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 87
  27. 2023VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Modelscited by 86
  28. 2023AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidancecited by 66
  29. 2023Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Modelscited by 58
  30. 2024Follow-Your-Click: Open-domain Regional Image Animation via Short Promptscited by 52

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 50; 2026: 36.

104 papers.

Sort:

By area:

  1. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  2. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  3. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  4. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  5. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 114
  6. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  7. 2025Controllable Video Generation: A Surveycited by 93
  8. 2025Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuningcited by 81
  9. 20243DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generationcited by 71
  10. 2025Follow-Your-Creation: Empowering 4D Creation through Video Inpaintingcited by 61
  11. 2024T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Designcited by 54
  12. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
38 more
  1. 2025FullDiT: Multi-Task Video Generative Foundation Model with Full Attentioncited by 45
  2. 2025CamCloneMaster: Enabling Reference-based Camera Control for Video Generationcited by 44
  3. 2024Video Motion Transfer with Diffusion Transformerscited by 39
  4. 2025Video-T1: Test-Time Scaling for Video Generationcited by 39
  5. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
  6. 2024VideoDirector: Precise Video Editing via Text-to-Video Modelscited by 36
  7. 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learningcited by 36
  8. 2024AnimateAnything: Consistent and Controllable Animation for Video Generationcited by 34
  9. 2025RelightVid: Temporal-Consistent Diffusion Model for Video Relightingcited by 28
  10. 2025VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generationcited by 25
  11. 2025FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarioscited by 25
  12. 2026FastVMT: Eliminating Redundancy in Video Motion Transfercited by 23
  13. 2025HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidancecited by 21
  14. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 18
  15. 2025VideoMage: Multi-Subject and Motion Customization of Text-to-Video Diffusion Modelscited by 17
  16. 2024TVG: A Training-free Transition Video Generation Method with Diffusion Modelscited by 15
  17. 2024X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Modelscited by 15
  18. 2025Free-Form Motion Control: Controlling the 6D Poses of Camera and Objects in Video Generationcited by 15
  19. 2025Decouple and Track: Benchmarking and Improving Video Diffusion Transformers for Motion Transfercited by 14
  20. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  21. 2025Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Controlcited by 14
  22. 2025TransPixeler: Advancing Text-to-Video Generation with Transparencycited by 12
  23. 2025ConMo: Controllable Motion Disentanglement and Recomposition for Zero-Shot Motion Transfercited by 12
  24. 2025Fast Multi-view Consistent 3D Editing with Video Priorscited by 12
  25. 2025BulletTime: Decoupled Control of Time and Camera Pose for Video Generationcited by 12
  26. 2026Physical Simulator In-the-Loop Video Generationcited by 12
  27. 2025Frame Guidance: Training-Free Guidance for Frame-Level Control in Video Diffusion Modelscited by 11
  28. 2024CPA: Camera-pose-awareness Diffusion Transformer for Video Generationcited by 10
  29. 2025DisMo: Disentangled Motion Representations for Open-World Motion Transfercited by 9
  30. 2026Plenoptic Video Generationcited by 9
  31. 2026SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generationcited by 9
  32. 2024Individual Content and Motion Dynamics Preserved Pruning for Video Diffusion Modelscited by 8
  33. 2025RelightMaster: Precise Video Relighting with Multi-plane Light Imagescited by 8
  34. 2024ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Waycited by 7
  35. 2025SynMotion: Semantic-Visual Adaptation for Motion Customized Video Generationcited by 7
  36. 2024Reenact Anything: Semantic Video Motion Transfer Using Motion-Textual Inversioncited by 6
  37. 2025DreamInsert: Zero-Shot Image-to-Video Object Insertion from A Single Imagecited by 6
  38. 2025IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generationcited by 6

50 most cited of 104 All 104 (JSON)