mlsanityData to 14 Oct 2026

Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Models

arXiv 2303.17599 ↗First posted 30 Mar 2023cs.CVcited by 137

On the map: Diffusion models → Video generation

Abstract (arXiv)

Large-scale text-to-image diffusion models achieve unprecedented success in image generation and editing. However, how to extend such success to video editing is unclear. Recent initial attempts at video editing require significant text-to-video data and computation resources for training, which is often not accessible. In this work, we propose vid2vid-zero, a simple yet effective method for zero-shot video editing. Our vid2vid-zero leverages off-the-shelf image diffusion models, and doesn't require training on any video. At the core of our method is a null-text inversion module for text-to-video alignment, a cross-frame modeling module for temporal consistency, and a spatial regularization module for fidelity to the original video. Without any training, we leverage the dynamic nature of the attention mechanism to enable bi-directional temporal modeling at test time. Experiments and analyses show promising results in editing attributes, subjects, places, etc., in real-world videos. Code is made available at urlhttps://github.com/baaivision/vid2vid-zero.

Key facts. Cited by 137 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 51 of its 52 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

51 of 52 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Pix2Video: Video Editing using Image Diffusion22 Mar
  2. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editing16 Mar
  3. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  4. 2023Dreamix: Video Diffusion Models are General Video Editors2 Feb
  5. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
  6. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov
  7. 2022Video Diffusion Models7 Apr

Other references · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  12. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
32 more
  1. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  5. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  7. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  8. 2021ViViT: A Video Vision Transformercited by 1,620
  9. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
  10. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  11. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  12. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  13. 2021Variational Diffusion Modelscited by 1,153
  14. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  15. 2022Token Merging: Your ViT But Fastercited by 1,021
  16. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  17. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  18. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  19. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 647
  20. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
  21. 2023Zero-shot Image-to-Image Translationcited by 505
  22. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  23. 2019Axial Attention in Multidimensional Transformerscited by 358
  24. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 314
  25. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 256
  26. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  27. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  28. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  29. 2019Dynamic Instance Normalization for Arbitrary Style Transfercited by 83
  30. 2023Edit-A-Video: Single Video Editing with Object-Aware Consistencycited by 67
  31. 2018Stroke Controllable Fast Style Transfer with Adaptive Receptive Fieldscited by 33
  32. 2022Learning Graph Neural Networks for Image Style Transfercited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 44; 2024: 47; 2025: 27; 2026: 19.

137 papers.

Sort:

By area:

  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  2. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  3. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  4. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  5. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  6. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  7. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 211
  8. 2023A Survey on Video Diffusion Modelscited by 206
  9. 2024Taming Rectified Flow for Inversion and Editingcited by 204
  10. 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 202
  11. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 195
  12. 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motioncited by 167
38 more
  1. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  2. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 126
  3. 2023RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Modelscited by 114
  4. 2023VidToMe: Video Token Merging for Zero-Shot Video Editingcited by 113
  5. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  6. 2023CCEdit: Creative and Controllable Video Editing via Diffusion Modelscited by 82
  7. 2023AVID: Any-Length Video Inpainting with Diffusion Modelcited by 82
  8. 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 81
  9. 2023FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesiscited by 79
  10. 2023MagicEdit: High-Fidelity and Temporally Coherent Video Editingcited by 76
  11. 2023Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animatorcited by 74
  12. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  13. 2023Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Modelscited by 59
  14. 2024COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editingcited by 59
  15. 2023MotionEditor: Editing Video Motion via Content-Aware Diffusioncited by 57
  16. 2023CVPR 2023 Text Guided Video Editing Competitioncited by 55
  17. 2024Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Modelscited by 55
  18. 2024I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Modelscited by 55
  19. 2025EditVerse: Unifying Image and Video Editing and Generation with In-Context Learningcited by 54
  20. 2023A Recipe for Scaling up Text-to-Video Generation with Text-free Videoscited by 51
  21. 2024FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translationcited by 49
  22. 2024FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Compositioncited by 49
  23. 2023Control-A-Video: Controllable Text-to-Video Diffusion Models with Motion Prior and Reward Feedback Learningcited by 48
  24. 2024UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editingcited by 48
  25. 2023InstructVid2Vid: Controllable Video Editing with Natural Language Instructionscited by 44
  26. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  27. 2024Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditionscited by 42
  28. 2024Video Editing via Factorized Diffusion Distillationcited by 41
  29. 2024Slicedit: Zero-Shot Video Editing With Text-to-Image Diffusion Models Using Spatio-Temporal Slicescited by 40
  30. 2023Fairy: Fast Parallelized Instruction-Guided Video-to-Video Synthesiscited by 39
  31. 2024Video Motion Transfer with Diffusion Transformerscited by 39
  32. 2024Looking Backward: Streaming Video-to-Video Translation with Feature Bankscited by 38
  33. 2023VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editingcited by 36
  34. 2024Generative Video Propagationcited by 36
  35. 2023Towards Consistent Video Editing with Text-to-Image Diffusion Modelscited by 34
  36. 2024Audio-Synchronized Visual Animationcited by 32
  37. 2024Video Diffusion Models: A Surveycited by 30
  38. 2024MotionFollower: Editing Video Motion via Lightweight Score-Guided Diffusioncited by 29

50 most cited of 137 All 137 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Investigating the Effectiveness of Cross-Attention to Unlock Zero-Shot Editing of Text-to-Video Diffusion Models
  2. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators
  3. 2025Are Image-to-Video Models Good Zero-Shot Image Editors?
  4. 2024Fine-gained Zero-shot Video Sampling
  5. 2024EVA: Zero-shot Accurate Attributes and Multi-Object Video Editing
  6. 2023EVE: Efficient zero-shot text-based Video Editing with Depth Map Guidance and Temporal Consistency Constraints
  7. 2026Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References
  8. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editing