# FastVMT: Eliminating Redundancy in Video Motion Transfer

arXiv: https://arxiv.org/abs/2602.05551 · HTML: https://mlsanity.com/paper/2602.05551/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 80 of its 99 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Builds on (most cited first, 30 of 80)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [Structure and Content-Guided Video Synthesis with Diffusion Models](https://mlsanity.com/paper/2302.03011/) (2023)
- [MotionCtrl: A Unified and Flexible Motion Controller for Video Generation](https://mlsanity.com/paper/2312.03641/) (2023)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [Latent Video Diffusion Models for High-Fidelity Long Video Generation](https://mlsanity.com/paper/2211.13221/) (2022)
- [Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos](https://mlsanity.com/paper/2304.01186/) (2023)
- [TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods](https://mlsanity.com/paper/2403.20150/) (2024)
- [Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2406.01900/) (2024)
- [MotionDirector: Motion Customization of Text-to-Video Diffusion Models](https://mlsanity.com/paper/2310.08465/) (2023)
- [SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generation](https://mlsanity.com/paper/2312.16272/) (2023)
- [Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance](https://mlsanity.com/paper/2306.00943/) (2023)
- [Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer](https://mlsanity.com/paper/2311.17009/) (2023)
- [EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer](https://mlsanity.com/paper/2503.07027/) (2025)
- [DUET: Dual Clustering Enhanced Multivariate Time Series Forecasting](https://mlsanity.com/paper/2412.10859/) (2024)
- [MotionClone: Training-Free Motion Cloning for Controllable Video Generation](https://mlsanity.com/paper/2406.05338/) (2024)
- [DiT4Edit: Diffusion Transformer for Image Editing](https://mlsanity.com/paper/2411.03286/) (2024)
- [Controllable Video Generation: A Survey](https://mlsanity.com/paper/2507.16869/) (2025)
- [MotionBooth: Motion-Aware Customized Text-to-Video Generation](https://mlsanity.com/paper/2406.17758/) (2024)
- [VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models](https://mlsanity.com/paper/2312.00845/) (2023)
- [Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuning](https://mlsanity.com/paper/2506.05207/) (2025)

## Built on by (most cited first, 23 of 23)

- [LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing](https://mlsanity.com/paper/2606.26740/) (2026)
- [Group Editing: Edit Multiple Images in One Go](https://mlsanity.com/paper/2603.22883/) (2026)
- [SCAIL-2: Unifying Controlled Character Animation with End-to-End In-Context Conditioning](https://mlsanity.com/paper/2606.10804/) (2026)
- [LiveLight: Real-time Streaming Video Relighting with Interactive Control](https://mlsanity.com/paper/2608.01771/) (2026)
- [UniFusion: A Unified Image Fusion Framework with Robust Representation and Source-Aware Preservation](https://mlsanity.com/paper/2603.14214/) (2026)
- [GlyphBanana: Advancing Precise Text Rendering Through Agentic Workflows](https://mlsanity.com/paper/2603.12155/) (2026)
- [Meta-CoT: Enhancing Granularity and Generalization in Image Editing](https://mlsanity.com/paper/2604.24625/) (2026)
- [MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models](https://mlsanity.com/paper/2602.13783/) (2026)
- [EasyVFX: Frequency-Driven Decoupling for Resource-Efficient VFX Generation](https://mlsanity.com/paper/2605.22051/) (2026)
- [InstanceAnimator: Multi-Instance Sketch Video Colorization](https://mlsanity.com/paper/2603.25357/) (2026)
- [GeoEdit: Geometry-Aware Object Editing via Dual-Branch Denoising](https://mlsanity.com/paper/2606.30003/) (2026)
- [Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models](https://mlsanity.com/paper/2607.09091/) (2026)
- [MSEditor: Toward Consistent Multi-Shot Video Editing](https://mlsanity.com/paper/2608.17559/) (2026)
- [LinCa: Accelerating Diffusion Models via Learnable Decomposed Feature Caching](https://mlsanity.com/paper/2608.17973/) (2026)
- [SubjectAnchor: Subject-Aware Memory-to-Video for Multi-Shot Storytelling](https://mlsanity.com/paper/2609.34502/) (2026)
- [Uni-Animator: Towards Unified Visual Colorization](https://mlsanity.com/paper/2602.23191/) (2026)
- [Video2LoRA: Unified Semantic-Controlled Video Generation via Per-Reference-Video LoRA](https://mlsanity.com/paper/2603.08210/) (2026)
- [Follow-Your-Preference++: Rethinking Preference Alignment for Image Inpainting](https://mlsanity.com/paper/2606.03216/) (2026)
- [Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization](https://mlsanity.com/paper/2606.26668/) (2026)
- [ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning](https://mlsanity.com/paper/2607.01677/) (2026)
- [MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation](https://mlsanity.com/paper/2607.14595/) (2026)
- [What Moves? Localized Motion Representations for Compositional Scene Control](https://mlsanity.com/paper/2609.04383/) (2026)
- [Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing](https://mlsanity.com/paper/2609.21268/) (2026)

## Abstract (arXiv)

Video motion transfer aims to synthesize videos by generating visual content according to a text prompt while transferring the motion pattern observed in a reference video. Recent methods predominantly use the Diffusion Transformer (DiT) architecture. To achieve satisfactory runtime, several methods attempt to accelerate the computations in the DiT, but fail to address structural sources of inefficiency. In this work, we identify and remove two types of computational redundancy in earlier work: motion redundancy arises because the generic DiT architecture does not reflect the fact that frame-to-frame motion is small and smooth; gradient redundancy occurs if one ignores that gradients change slowly along the diffusion trajectory. To mitigate motion redundancy, we mask the corresponding attention layers to a local neighborhood such that interaction weights are not computed unnecessarily distant image regions. To exploit gradient redundancy, we design an optimization scheme that reuses gradients from previous diffusion steps and skips unwarranted gradient computations. On average, FastVMT achieves a 3.43x speedup without degrading the visual fidelity or the temporal consistency of the generated videos.