mlsanityData to 14 Oct 2026

Motion-Conditioned Image Animation for Video Editing

arXiv 2311.18827 ↗First posted 30 Nov 2023cs.GR cs.AI cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

We introduce MoCA, a Motion-Conditioned Image Animation approach for video editing. It leverages a simple decomposition of the video editing problem into image editing followed by motion-conditioned image animation. Furthermore, given the lack of robust evaluation datasets for video editing, we introduce a new benchmark that measures edit capability across a wide variety of tasks, such as object replacement, background changes, style changes, and motion edits. We present a comprehensive human evaluation of the latest video editing methods along with MoCA, on our proposed benchmark. MoCA establishes a new state-of-the-art, demonstrating greater human preference win-rate, and outperforming notable recent approaches including Dreamix (63%), MasaCtrl (75%), and Tune-A-Video (72%), with especially significant improvements for motion edits.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 42 of its 44 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

42 of 44 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
30 more
  1. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  2. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  3. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  4. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  6. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  7. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  8. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  10. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  11. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  12. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  13. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  14. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  15. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  16. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  17. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  18. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  19. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  20. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
  21. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 327
  22. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  23. 2023Pix2Video: Video Editing using Image Diffusioncited by 320
  24. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 314
  25. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 255
  26. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  27. 2023Dreamix: Video Diffusion Models are General Video Editorscited by 191
  28. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  29. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  30. 2023LEDITS: Real Image Editing with DDPM Inversion and Semantic Guidancecited by 49

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 11; 2026: 4.

26 papers.

Sort:

By area:

  1. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  2. 2024I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Modelscited by 55
  3. 2024UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editingcited by 48
  4. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  5. 2024Video Editing via Factorized Diffusion Distillationcited by 41
  6. 2024Generative Video Propagationcited by 36
  7. 2025Generating Multi-Image Synthetic Data for Text-to-Image Customizationcited by 25
  8. 2024Compositional 3D-aware Video Generation with LLM Directorcited by 18
  9. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  10. 2025Learning an Image Editing Model without Image Editing Pairscited by 13
  11. 2025Fast Multi-view Consistent 3D Editing with Video Priorscited by 12
  12. 2024MotiF: Making Text Count in Image Animation with Motion Focal Losscited by 10
14 more
  1. 2025VidSketch: Hand-drawn Sketch-Driven Video Generation with Diffusion Controlcited by 9
  2. 2024One missing piece in Vision and Language: A Survey on Comics Understandingcited by 8
  3. 2026PAI-Studio: Cinematic Video Background Replacement with Camera-Aware Motioncited by 8
  4. 2025Target-Aware Video Diffusion Modelscited by 7
  5. 2026Evolution of Video Generative Foundationscited by 7
  6. 2024Reenact Anything: Semantic Video Motion Transfer Using Motion-Textual Inversioncited by 6
  7. 2025Taming Flow-based I2V Models for Creative Video Editingcited by 5
  8. 2024Beyond Generation: Unlocking Universal Editing via Self-Supervised Fine-Tuningcited by 2
  9. 2026Towards 3D-Aware Video Diffusion Models: Render-Free Human Motion Control with Mesh Tokenizationcited by 1
  10. 2025Edit as You See: Image-guided Video Editing via Masked Motion Modelingcited by 0
  11. 2025EndoControlMag: Robust Endoscopic Vascular Motion Magnification with Periodic Reference Resetting and Hierarchical Tissue-aware Dual-Mask Controlcited by 0
  12. 2025VRWKV-Editor: Reducing quadratic complexity in transformer-based video editingcited by 0
  13. 2025VIDMP3: Video Editing by Representing Motion with Pose and Position Priorscited by 0
  14. 2026Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomycited by 0