mlsanityData to 7 Oct 2026

AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Tasks

arXiv 2403.14468 ↗First posted 21 Mar 2024cs.CV cs.AI cs.MMcited by 151

Not on the map. Why?

Cited by 48 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In the dynamic field of digital content creation using generative models, state-of-the-art video editing models still do not offer the level of quality and control that users desire. Previous works on video editing either extended from image-based generative models in a zero-shot manner or necessitated extensive fine-tuning, which can hinder the production of fluid video edits. Furthermore, these methods frequently rely on textual input as the editing guidance, leading to ambiguities and limiting the types of edits they can perform. Recognizing these challenges, we introduce AnyV2V, a novel tuning-free paradigm designed to simplify video editing into two primary steps: (1) employing an off-the-shelf image editing model to modify the first frame, (2) utilizing an existing image-to-video generation model to generate the edited video through temporal feature injection. AnyV2V can leverage any existing image editing tools to support an extensive array of video editing tasks, including prompt-based editing, reference-based style transfer, subject-driven editing, and identity manipulation, which were unattainable by previous methods. AnyV2V can also support any video length. Our evaluation shows that AnyV2V achieved CLIP-scores comparable to other baseline methods. Furthermore, AnyV2V significantly outperformed these baselines in human evaluations, demonstrating notable improvements in visual consistency with the source video while producing high-quality edits across all editing tasks.

Key facts. Cited by 151 papers in the map (data through 7 Oct 2026). Builds on 51 of its 57 references that are in the map. First posted on arXiv 21 Mar 2024.

Builds on

51 of 57 references.

References · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  10. 2022Video Diffusion Modelscited by 2,280
  11. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  12. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
38 more
  1. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  2. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  3. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  4. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  5. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  6. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  7. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  8. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  9. 2015A Neural Algorithm of Artistic Stylecited by 480
  10. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  11. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  12. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  13. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  14. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  15. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
  16. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  17. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  18. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
  19. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 324
  20. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  21. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  22. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  23. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  24. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 231
  25. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 173
  26. 2023FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editingcited by 165
  27. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  28. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  29. 2023FreeInit: Bridging Initialization Gap in Video Diffusion Modelscited by 93
  30. 2017Exploring the structure of a real-time, arbitrary neural artistic stylization networkcited by 89
  31. 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 79
  32. 2023FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesiscited by 79
  33. 2023ImagenHub: Standardizing the evaluation of conditional image generation modelscited by 75
  34. 2023VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondencecited by 69
  35. 2023Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Modelscited by 58
  36. 2024UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editingcited by 47
  37. 2023Fairy: Fast Parallelized Instruction-Guided Video-to-Video Synthesiscited by 39
  38. 2022WISE: Whitebox Image Stylization by Example-based Learningcited by 5

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 63; 2026: 75.

151 papers.

Sort:

By area:

  1. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
  2. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  3. 2025OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editingcited by 50
  4. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  5. 2025VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Controlcited by 42
  6. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40
  7. 2024Video Motion Transfer with Diffusion Transformerscited by 39
  8. 2025VEGGIE: Instructional Editing and Reasoning Video Concepts with Grounded Generationcited by 39
  9. 2024Looking Backward: Streaming Video-to-Video Translation with Feature Bankscited by 38
  10. 2025In-Context Learning with Unpaired Clips for Instruction-based Video Editingcited by 34
  11. 2026Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidancecited by 34
  12. 2025InstructX: Towards Unified Visual Editing with MLLM Guidancecited by 31
38 more
  1. 2025LoRA-Edit: Controllable First-Frame-Guided Video Editing via Mask-Aware LoRA Fine-Tuningcited by 27
  2. 2025ContextFlow: Training-Free Video Object Editing via Adaptive Context Enrichmentcited by 27
  3. 2025IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessmentcited by 27
  4. 2024StableV2V: Stablizing Shape Consistency in Video-to-Video Editingcited by 25
  5. 2025Region-Constraint In-Context Generation for Instructional Video Editingcited by 22
  6. 2025Generative Video Motion Editing with 3D Point Trackscited by 21
  7. 2026Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editingcited by 18
  8. 2024DIVE: Taming DINO for Subject-Driven Video Editingcited by 17
  9. 2025VIVA: VLM-Guided Instruction-Based Video Editing with Reward Optimizationcited by 16
  10. 2025FiVE: A Fine-grained Video Editing Benchmark for Evaluating Emerging Diffusion and Rectified Flow Modelscited by 15
  11. 2025SketchVideo: Sketch-based Video Generation and Editingcited by 15
  12. 2025ConMo: Controllable Motion Disentanglement and Recomposition for Zero-Shot Motion Transfercited by 12
  13. 2024CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusioncited by 11
  14. 2025Controllable Weather Synthesis and Removal with Video Diffusion Modelscited by 11
  15. 2025Light-X: Generative 4D Video Rendering with Camera and Illumination Controlcited by 11
  16. 2025EasyV2V: A High-quality Instruction-based Video Editing Frameworkcited by 11
  17. 2026Lance: Unified Multimodal Modeling by Multi-Task Synergycited by 11
  18. 2026Bernini: Latent Semantic Planning for Video Diffusioncited by 11
  19. 2024EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Modelscited by 10
  20. 2025Frame In-N-Out: Unbounded Controllable Image-to-Video Generationcited by 10
  21. 2025DynVFX: Augmenting Real Videos with Dynamic Contentcited by 9
  22. 2025Tinker: Diffusion's Gift to 3D--Multi-View Consistent Editing From Sparse Inputs without Per-Scene Optimizationcited by 9
  23. 2025OmniInsert: Mask-Free Video Insertion of Any Reference via Diffusion Transformer Modelscited by 9
  24. 2025WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generationcited by 9
  25. 2025Shape-for-Motion: Precise and Consistent Video Editing with 3D Proxycited by 8
  26. 2025RelightMaster: Precise Video Relighting with Multi-plane Light Imagescited by 8
  27. 2026Context Unrolling in Omni Modelscited by 8
  28. 2026StreamingEffect: Real-Time Human-Centric Video Effect Generationcited by 8
  29. 2026VISTA: Triplet-Supervised Video Style Transfer with Diffusion Transformerscited by 8
  30. 2025V2Edit: Versatile Video Diffusion Editor for Videos and 3D Scenescited by 7
  31. 2025InstructVEdit: A Holistic Approach for Instructional Video Editingcited by 7
  32. 2025Understanding Attention Mechanism in Video Diffusion Modelscited by 7
  33. 2025Consistent Video Editing as Flow-Driven Image-to-Video Generationcited by 7
  34. 2025EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editingcited by 7
  35. 2026PAI-Studio: Cinematic Video Background Replacement with Camera-Aware Motioncited by 7
  36. 2025IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generationcited by 6
  37. 2025ASTRA: Let Arbitrary Subjects Transform in Video Editingcited by 6
  38. 2025FreeViS: Training-free Video Stylization with Inconsistent Referencescited by 6

50 most cited of 151 All 151 (JSON)