mlsanityData to 14 Oct 2026

Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuning

arXiv 2506.05207 ↗First posted 5 Jun 2025cs.CVcited by 81

On the map: Diffusion models → Video generation

Abstract (arXiv)

Recently, breakthroughs in the video diffusion transformer have shown remarkable capabilities in diverse motion generations. As for the motion-transfer task, current methods mainly use two-stage Low-Rank Adaptations (LoRAs) finetuning to obtain better performance. However, existing adaptation-based motion transfer still suffers from motion inconsistency and tuning inefficiency when applied to large video diffusion transformers. Naive two-stage LoRA tuning struggles to maintain motion consistency between generated and input videos due to the inherent spatial-temporal coupling in the 3D attention operator. Additionally, they require time-consuming fine-tuning processes in both stages. To tackle these issues, we propose Follow-Your-Motion, an efficient two-stage video motion transfer framework that finetunes a powerful video diffusion transformer to synthesize complex motion. Specifically, we propose a spatial-temporal decoupled LoRA to decouple the attention architecture for spatial appearance and temporal motion processing. During the second training stage, we design the sparse motion sampling and adaptive RoPE to accelerate the tuning speed. To address the lack of a benchmark for this field, we introduce MotionBench, a comprehensive benchmark comprising diverse motion, including creative camera motion, single object motion, multiple object motion, and complex human motion. We show extensive evaluations on MotionBench to verify the superiority of Follow-Your-Motion.

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 52 of its 73 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

52 of 73 references.

Closest ancestors · 22

Same direction, newest first

  1. 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing24 Feb
  2. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity3 Feb
  3. 2025FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors14 Jan
  4. 2024Video Motion Transfer with Diffusion Transformers10 Dec
  5. 2024MotionShop: Zero-Shot Motion Transfer in Video Diffusion Models with Mixture of Score Guidance6 Dec
  6. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  7. 2024COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editing13 Jun
  8. 2024EasyAnimate: High-Performance Video Generation Framework with Hybrid Windows Attention and Reward Backpropagation29 May
  9. 2024Video Diffusion Models are Training-free Motion Interpreter and Controller23 May
  10. 2024MotionMaster: Training-free Camera Motion Transfer For Video Generation24 Apr
  11. 2024DreamMotion: Space-Time Self-Similar Score Distillation for Zero-Shot Video Editing18 Mar
  12. 2024Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models22 Feb
10 more
  1. 2023MotionCrafter: One-Shot Motion Customization of Diffusion Models8 Dec
  2. 2023VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models1 Dec
  3. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors18 Oct
  4. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Models12 Oct
  5. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editing18 Aug
  6. 2023ModelScope Text-to-Video Technical Report12 Aug
  7. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  8. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editing16 Mar
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
  10. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov

Other references · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  5. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  6. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  7. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
  8. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
  9. 2024Taming Rectified Flow for Inversion and Editingcited by 204
  10. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 174
  11. 2023Make-Your-Video: Customized Video Generation Using Textual and Structural Guidancecited by 129
  12. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
18 more
  1. 2024MotionClone: Training-Free Motion Cloning for Controllable Video Generationcited by 105
  2. 2024DiT4Edit: Diffusion Transformer for Image Editingcited by 101
  3. 2025CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generationcited by 74
  4. 2023MagicStick: Controllable Video Editing via Control Handle Transformationscited by 62
  5. 2024Follow-Your-Click: Open-domain Regional Image Animation via Short Promptscited by 52
  6. 2024MultiBooth: Towards Generating All Your Concepts in an Image from Textcited by 45
  7. 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
  8. 2024Follow-Your-Canvas: Higher-Resolution Video Outpainting with Extensive Content Generationcited by 42
  9. 2024InstantSwap: Fast Customized Concept Swapping across Sharp Shape Differencescited by 34
  10. 2024Motion Inversion for Video Customizationcited by 30
  11. 2024CharacterFactory: Sampling Consistent Characters with GANs for Diffusion Modelscited by 24
  12. 2025Dynamic Concepts Personalization from Single Videoscited by 20
  13. 2023M-BEV: Masked BEV Perception for Robust Autonomous Drivingcited by 19
  14. 2025AvatarArtist: Open-Domain 4D Avatarizationcited by 15
  15. 2024UniPaint: Unified Space-time Video Inpainting via Mixture-of-Expertscited by 13
  16. 2025Follow-Your-Color: Multi-Instance Sketch Colorizationcited by 10
  17. 2024Follow-Your-MultiPose: Tuning-Free Multi-Character Text-to-Video Generation via Pose Guidancecited by 7
  18. 2025Enhancing Image Generation Fidelity via Progressive Promptscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 35; 2026: 46.

81 papers.

Sort:

By area:

  1. 2025Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animationcited by 43
  2. 2025Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Modelscited by 36
  3. 2025MultiShotMaster: A Controllable Multi-Shot Video Generation Frameworkcited by 36
  4. 2025Aurora: Towards Universal Generative Multimodal Time Series Forecastingcited by 29
  5. 2025Follow-Your-Shape: Shape-Aware Image Editing via Trajectory-Guided Region Controlcited by 28
  6. 2025ContextFlow: Training-Free Video Object Editing via Adaptive Context Enrichmentcited by 27
  7. 2026FastVMT: Eliminating Redundancy in Video Motion Transfercited by 23
  8. 2025Hybrid Token Compression for Vision-Language Modelscited by 20
  9. 2025Top-Down Semantic Refinement for Image Captioningcited by 16
  10. 2026LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editingcited by 15
  11. 2025Mitty: Diffusion-based Human-to-Robot Video Generationcited by 14
  12. 2025OmniPSD: Layered PSD Generation with Diffusion Transformercited by 13
38 more
  1. 2026GCGNet: Graph-Consistent Generative Network for Time Series Forecasting with Exogenous Variablescited by 13
  2. 2025Mitigating Error Accumulation in Co-Speech Motion Generation via Global Rotation Diffusion and Multi-Level Constraintscited by 12
  3. 2026VideoWorld 2: Learning Transferable Knowledge from Real-world Videoscited by 12
  4. 2025SkipVAR: Accelerating Visual Autoregressive Modeling via Adaptive Frequency-Aware Skippingcited by 11
  5. 2026Group Editing: Edit Multiple Images in One Gocited by 11
  6. 2025Follow-Your-Instruction: A Comprehensive MLLM Agent for World Data Synthesiscited by 10
  7. 2025SMRABooth: Subject and Motion Representation Alignment for Customized Video Generationcited by 10
  8. 2025Zo3T: Zero-Shot 3D-Aware Trajectory-Guided Image-to-Video Generation via Test-Time Trainingcited by 9
  9. 2025UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesiscited by 8
  10. 2025ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecastingcited by 8
  11. 2025VideoCoF: Unified Video Editing with Temporal Reasonercited by 8
  12. 2026PAI-Studio: Cinematic Video Background Replacement with Camera-Aware Motioncited by 8
  13. 2026SCAIL-2: Unifying Controlled Character Animation with End-to-End In-Context Conditioningcited by 8
  14. 2025CML-Bench: A Framework for Evaluating and Enhancing LLM-Powered Movie Scripts Generationcited by 7
  15. 2026LiveLight: Real-time Streaming Video Relighting with Interactive Controlcited by 7
  16. 2025LongAnimation: Long Animation Generation with Dynamic Global-Local Memorycited by 6
  17. 2026OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfercited by 6
  18. 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generationcited by 6
  19. 2025GR-Gaussian: Graph-Based Radiative Gaussian Splatting for Sparse-View CT Reconstructioncited by 5
  20. 2025Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceiverscited by 5
  21. 2025OmniRefiner: Reinforcement-Guided Local Diffusion Refinementcited by 5
  22. 2025The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignmentcited by 5
  23. 2026FFP-300K: Scaling First-Frame Propagation for Generalizable Video Editingcited by 5
  24. 2026Embedding-perturbed Exploration Preference Optimization for Flow Modelscited by 5
  25. 2025NumeriKontrol: Adding Numeric Control to Diffusion Transformers for Instruction-based Image Editingcited by 4
  26. 2025IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learningcited by 4
  27. 2026GlyphBanana: Advancing Precise Text Rendering Through Agentic Workflowscited by 4
  28. 2026OmniHumanoid: Streaming Cross-Embodiment Video Generation with Paired-Free Adaptationcited by 4
  29. 2025MultiMotion: Multi Subject Video Motion Transfer via Video Diffusion Transformercited by 3
  30. 2025Active Intelligence in Video Avatars via Closed-loop World Modelingcited by 3
  31. 2026Let Your Image Move with Your Motion! -- Implicit Multi-Object Multi-Motion Transfercited by 3
  32. 2026Towards Multimodal Time Series Anomaly Detection with Semantic Alignment and Condensed Interactioncited by 3
  33. 2025ChronosObserver: Taming 4D World with Hyperspace Diffusion Samplingcited by 2
  34. 2026SIGMA: Selective-Interleaved Generation with Multi-Attribute Tokenscited by 2
  35. 2026MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Modelscited by 2
  36. 2026AgriWorld:A World Tools Protocol Framework for Verifiable Agricultural Reasoning with Code-Executing LLM Agentscited by 2
  37. 2026ResWM: Residual-Action World Model for Visual RLcited by 2
  38. 2026Focused Forcing: Content-Aware Per-Frame KV Selection for Efficient Autoregressive Video Diffusioncited by 2

50 most cited of 81 All 81 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Decouple and Track: Benchmarking and Improving Video Diffusion Transformers for Motion Transfer
  2. 2026Controlling Motion Transfer in Diffusion Transformers via Attention Heads
  3. 2024MotionFlow: Attention-Driven Motion Transfer in Video Diffusion Models
  4. 2025LiON-LoRA: Rethinking LoRA Fusion to Unify Controllable Spatial and Temporal Generation for Video Diffusion
  5. 2025LMP: Leveraging Motion Prior in Zero-Shot Video Generation with Diffusion Transformer
  6. 2024Edit-Your-Motion: Space-Time Diffusion Decoupling Learning for Video Motion Editing
  7. 2026QWERTY: Training-Free Motion Control via Query-Warped Video Diffusion Transformers
  8. 2026FrameDiT: Diffusion Transformer with Matrix Attention for Efficient Video Generation