# MotionEditor: Editing Video Motion via Content-Aware Diffusion

arXiv: https://arxiv.org/abs/2311.18830 · HTML: https://mlsanity.com/paper/2311.18830/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 50 of its 59 references that are in the map.
- First posted on arXiv 30 Nov 2023.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 50)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2212.11565/) (2022)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators](https://mlsanity.com/paper/2303.13439/) (2023)
- [FateZero: Fusing Attentions for Zero-shot Text-based Video Editing](https://mlsanity.com/paper/2303.09535/) (2023)
- [ControlVideo: Training-free Controllable Text-to-Video Generation](https://mlsanity.com/paper/2305.13077/) (2023)

## Built on by (most cited first, 30 of 57)

- [Training-Free Consistent Text-to-Image Generation](https://mlsanity.com/paper/2402.03286/) (2024)
- [StableAnimator: High-Quality Identity-Preserving Human Image Animation](https://mlsanity.com/paper/2411.17697/) (2024)
- [MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidance](https://mlsanity.com/paper/2503.16421/) (2025)
- [UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editing](https://mlsanity.com/paper/2402.13185/) (2024)
- [TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control](https://mlsanity.com/paper/2410.10133/) (2024)
- [Diffusion Model-Based Video Editing: A Survey](https://mlsanity.com/paper/2407.07111/) (2024)
- [StableAvatar: Infinite-Length Audio-Driven Avatar Video Generation](https://mlsanity.com/paper/2508.08248/) (2025)
- [VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Control](https://mlsanity.com/paper/2501.01427/) (2025)
- [TrackGo: A Flexible and Efficient Method for Controllable Video Generation](https://mlsanity.com/paper/2408.11475/) (2024)
- [MotionFollower: Editing Video Motion via Lightweight Score-Guided Diffusion](https://mlsanity.com/paper/2405.20325/) (2024)
- [OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning](https://mlsanity.com/paper/2603.24458/) (2026)
- [VIVID-10M: A Dataset and Baseline for Versatile and Interactive Video Local Editing](https://mlsanity.com/paper/2411.15260/) (2024)
- [FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios](https://mlsanity.com/paper/2505.03730/) (2025)
- [DreamRelation: Relation-Centric Video Customization](https://mlsanity.com/paper/2503.07602/) (2025)
- [Generative Video Motion Editing with 3D Point Tracks](https://mlsanity.com/paper/2512.02015/) (2025)
- [AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Prediction](https://mlsanity.com/paper/2406.06465/) (2024)
- [EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation](https://mlsanity.com/paper/2503.15831/) (2025)
- [FiVE: A Fine-grained Video Editing Benchmark for Evaluating Emerging Diffusion and Rectified Flow Models](https://mlsanity.com/paper/2503.13684/) (2025)
- [Simulating the Real World: A Unified Survey of Multimodal Generative Models](https://mlsanity.com/paper/2503.04641/) (2025)
- [REDUCIO! Generating 1K Video within 16 Seconds using Extremely Compressed Motion Latents](https://mlsanity.com/paper/2411.13552/) (2024)
- [StableAnimator++: Overcoming Pose Misalignment and Face Distortion for Human Image Animation](https://mlsanity.com/paper/2507.15064/) (2025)
- [FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors](https://mlsanity.com/paper/2501.08225/) (2025)
- [ConMo: Controllable Motion Disentanglement and Recomposition for Zero-Shot Motion Transfer](https://mlsanity.com/paper/2504.02451/) (2025)
- [DualCamCtrl: Dual-Branch Diffusion Model for Geometry-Aware Camera-Controlled Video Generation](https://mlsanity.com/paper/2511.23127/) (2025)
- [Edit-Your-Motion: Space-Time Diffusion Decoupling Learning for Video Motion Editing](https://mlsanity.com/paper/2405.04496/) (2024)
- [Versatile Transition Generation with Image-to-Video Diffusion](https://mlsanity.com/paper/2508.01698/) (2025)
- [Future Optical Flow Prediction Improves Robot Control & Video Generation](https://mlsanity.com/paper/2601.10781/) (2026)
- [Reenact Anything: Semantic Video Motion Transfer Using Motion-Textual Inversion](https://mlsanity.com/paper/2408.00458/) (2024)
- [C-Drag: Chain-of-Thought Driven Motion Controller for Video Generation](https://mlsanity.com/paper/2502.19868/) (2025)
- [H2OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformers](https://mlsanity.com/paper/2509.06956/) (2025)

## Abstract (arXiv)

Existing diffusion-based video editing models have made gorgeous advances for editing attributes of a source video over time but struggle to manipulate the motion information while preserving the original protagonist's appearance and background. To address this, we propose MotionEditor, a diffusion model for video motion editing. MotionEditor incorporates a novel content-aware motion adapter into ControlNet to capture temporal motion correspondence. While ControlNet enables direct generation based on skeleton poses, it encounters challenges when modifying the source motion in the inverted noise due to contradictory signals between the noise (source) and the condition (reference). Our adapter complements ControlNet by involving source content to transfer adapted control signals seamlessly. Further, we build up a two-branch architecture (a reconstruction branch and an editing branch) with a high-fidelity attention injection mechanism facilitating branch interaction. This mechanism enables the editing branch to query the key and value from the reconstruction branch in a decoupled manner, making the editing branch retain the original background and protagonist appearance. We also propose a skeleton alignment algorithm to address the discrepancies in pose size and position. Experiments demonstrate the promising motion editing ability of MotionEditor, both qualitatively and quantitatively.