mlsanityData to 14 Oct 2026

RoPECraft: Training-Free Motion Transfer with Trajectory-Guided RoPE Optimization on Diffusion Transformers

arXiv 2505.13344 ↗First posted 19 May 2025cs.CV cs.AI cs.LGcited by 14

On the map: Diffusion models → Video generation

Abstract (arXiv)

We propose RoPECraft, a training-free video motion transfer method for diffusion transformers that operates solely by modifying their rotary positional embeddings (RoPE). We first extract dense optical flow from a reference video, and utilize the resulting motion offsets to warp the complex-exponential tensors of RoPE, effectively encoding motion into the generation process. These embeddings are then further optimized during denoising time steps via trajectory alignment between the predicted and target velocities using a flow-matching objective. To keep the output faithful to the text prompt and prevent duplicate generations, we incorporate a regularization term based on the phase components of the reference video's Fourier transform, projecting the phase angles onto a smooth manifold to suppress high-frequency artifacts. Experiments on benchmarks reveal that RoPECraft outperforms all recently published methods, both qualitatively and quantitatively.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 48 of its 51 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

48 of 51 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise5 Feb
  2. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noise14 Jan
  3. 2024Video Motion Transfer with Diffusion Transformers10 Dec
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  5. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generation31 Jul
  6. 2024Video Diffusion Models are Training-free Motion Interpreter and Controller23 May
  7. 2024Lumiere: A Space-Time Diffusion Model for Video Generation23 Jan
  8. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
  9. 2024Latte: Latent Diffusion Transformer for Video Generation5 Jan
  10. 2023VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models1 Dec
  11. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  12. 2023ModelScope Text-to-Video Technical Report12 Aug
2 more
  1. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  2. 2022Video Diffusion Models7 Apr

Other references · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2024The Llama 3 Herd of Modelscited by 14,715
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2022Flow Matching for Generative Modelingcited by 5,702
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  12. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
22 more
  1. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  3. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  4. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  5. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  6. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  7. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
  8. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  9. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  10. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
  11. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  12. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 264
  13. 2024Rotary Position Embedding for Vision Transformercited by 236
  14. 2023Make-Your-Video: Customized Video Generation Using Textual and Structural Guidancecited by 129
  15. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  16. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 115
  17. 2023AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidancecited by 66
  18. 2024ViTAR: Vision Transformer with Any Resolutioncited by 25
  19. 2025VRoPE: Rotary Position Embedding for Video Large Language Modelscited by 18
  20. 2025RealCam-Vid: High-resolution Video Dataset with Dynamic Scenes and Metric-scale Camera Movementscited by 14
  21. 2025ConMo: Controllable Motion Disentanglement and Recomposition for Zero-Shot Motion Transfercited by 12
  22. 2024Reenact Anything: Semantic Video Motion Transfer Using Motion-Textual Inversioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 12.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026RoPeSLR: 3D RoPE-driven Sparse-LowRank Attention for Efficient Diffusion Transformers
  2. 2026AcrossVAM2.0: Conflict-Aware Point Supervision for Parameter-Efficient Video Diffusion Adaptation
  3. 2025Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuning
  4. 2024FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modification
  5. 2026GeoShrink: Accelerating Diffusion Transformers with Two Lines of Code
  6. 2026Not All Tokens Need 40 Steps: Heterogeneous Step Allocation in Diffusion Transformers for Efficient Video Generation
  7. 2025Decouple and Track: Benchmarking and Improving Video Diffusion Transformers for Motion Transfer
  8. 2026ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformer