mlsanityData to 7 Oct 2026

Videoshop: Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversion

arXiv 2403.14617 ↗First posted 21 Mar 2024cs.CV cs.AI cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

We introduce Videoshop, a training-free video editing algorithm for localized semantic edits. Videoshop allows users to use any editing software, including Photoshop and generative inpainting, to modify the first frame; it automatically propagates those changes, with semantic, spatial, and temporally consistent motion, to the remaining frames. Unlike existing methods that enable edits only through imprecise textual instructions, Videoshop allows users to add or remove objects, semantically change objects, insert stock photos into videos, etc. with fine-grained control over locations and appearance. We achieve this through image-based video editing by inverting latents with noise extrapolation, from which we generate videos conditioned on the edited image. Videoshop produces higher quality edits against 6 baselines on 2 editing benchmarks using 10 evaluation metrics.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 50 of its 62 references that are in the map. First posted on arXiv 21 Mar 2024.

Builds on

50 of 62 references.

References · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  7. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  8. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  9. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  10. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  11. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  12. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
38 more
  1. 2023ModelScope Text-to-Video Technical Reportcited by 651
  2. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  3. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  4. 2023CoTracker: It is Better to Track Togethercited by 516
  5. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  6. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  7. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  8. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  9. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  10. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
  11. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  12. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  13. 2018Everybody Dance Nowcited by 313
  14. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  15. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 256
  16. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  17. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  18. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
  19. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  20. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  21. 2023RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Modelscited by 113
  22. 2024Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesiscited by 105
  23. 2023Edit-A-Video: Single Video Editing with Object-Aware Consistencycited by 65
  24. 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 62
  25. 2023Shadows Don't Lie and Lines Can't Bend! Generative Models don't know Projective Geometry...for nowcited by 61
  26. 2023Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Modelscited by 58
  27. 2024Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Modelscited by 55
  28. 2023Generative Models: What Do They Know? Do They Know Things? Let's Find Out!cited by 50
  29. 2024Image Sculpting: Precise Object Editing with 3D Geometry Controlcited by 45
  30. 2023StyleGAN knows Normal, Depth, Albedo, and Morecited by 44
  31. 2023Exact Diffusion Inversion via Bi-directional Integration Approximationcited by 43
  32. 2023VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editingcited by 36
  33. 20233D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillationcited by 34
  34. 20223D Highlighter: Localizing Regions on 3D Shapes via Text Descriptionscited by 32
  35. 2024Object-Centric Diffusion for Efficient Video Editingcited by 31
  36. 2022Dance Style Transfer with Cross-modal Transformercited by 20
  37. 2020Cut-and-Paste Object Insertion by Enabling Deep Image Prior for Reshadingcited by 12
  38. 2023DiffusionAtlas: High-Fidelity Consistent Diffusion Video Editingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 17; 2026: 6.

27 papers.

Sort:

By area:

  1. 2024Taming Rectified Flow for Inversion and Editingcited by 200
  2. 2024Not All Noises Are Created Equally:Diffusion Noise Selection and Optimizationcited by 70
  3. 2025InsViE-1M: Effective Instruction-based Video Editing with Elaborate Dataset Constructioncited by 62
  4. 2024Generative Video Propagationcited by 36
  5. 2025In-Context Learning with Unpaired Clips for Instruction-based Video Editingcited by 34
  6. 2025VIVA: VLM-Guided Instruction-Based Video Editing with Reward Optimizationcited by 16
  7. 2025Zero-Shot Audio-Visual Editing via Cross-Modal Delta Denoisingcited by 13
  8. 2025Fast Multi-view Consistent 3D Editing with Video Priorscited by 12
  9. 2025Morpheus: Text-Driven 3D Gaussian Splat Shape and Color Stylizationcited by 8
  10. 2025Visual Prompting for One-shot Controllable Video Editing without Inversioncited by 8
  11. 2025Shape-for-Motion: Precise and Consistent Video Editing with 3D Proxycited by 8
  12. 2025V2Edit: Versatile Video Diffusion Editor for Videos and 3D Scenescited by 7
15 more
  1. 2025Taming Flow-based I2V Models for Creative Video Editingcited by 5
  2. 2025Coherent Audio-Visual Editing via Conditional Audio Generation Following Video Editscited by 5
  3. 2026FFP-300K: Scaling First-Frame Propagation for Generalizable Video Editingcited by 5
  4. 2026Omni123: Exploring 3D Native Foundation Models with Limited 3D Data by Unifying Text to 2D and 3D Generationcited by 3
  5. 2026SteerVTE: Seamless Video Text Editing with Style and Glyph Controlcited by 3
  6. 2024Beyond Generation: Unlocking Universal Editing via Self-Supervised Fine-Tuningcited by 2
  7. 2025SPICE: A Synergistic, Precise, Iterative, and Customizable Image Editing Workflowcited by 2
  8. 2025RefTok: Reference-Based Tokenization for Video Generationcited by 2
  9. 2025Dress&Dance: Dress up and Dance as You Like It - Technical Previewcited by 2
  10. 2025FAME: Fairness-aware Attention-modulated Video Editingcited by 2
  11. 2025GIFT: Generated Indoor video frames for Texture-less point trackingcited by 0
  12. 2025VALA: Learning Latent Anchors for Training-Free and Temporally Consistentcited by 0
  13. 2026FREE-Edit: Using Editing-aware Injection in Rectified Flow Models for Zero-shot Image-Driven Video Editingcited by 0
  14. 2026Tuning-free Instruction-based Video Editing Via Structural Noise Initialization and Guidancecited by 0
  15. 2026Timestep Rescheduling in Diffusion Inversioncited by 0