mlsanityData to 7 Oct 2026

StableVideo: Text-driven Consistency-aware Diffusion Video Editing

arXiv 2308.09592 ↗First posted 18 Aug 2023cs.CVcited by 182

On the map: Diffusion models → Video generation

Abstract (arXiv)

Diffusion-based methods can generate realistic images and videos, but they struggle to edit existing objects in a video while preserving their appearance over time. This prevents diffusion models from being applied to natural video editing in practical scenarios. In this paper, we tackle this problem by introducing temporal dependency to existing text-driven diffusion models, which allows them to generate consistent appearance for the edited objects. Specifically, we develop a novel inter-frame propagation mechanism for diffusion video editing, which leverages the concept of layered representations to propagate the appearance information from one frame to the next. We then build up a text-driven video editing framework based on this mechanism, namely StableVideo, which can achieve consistency-aware video editing. Extensive experiments demonstrate the strong editing capability of our approach. Compared with state-of-the-art video editing methods, our approach shows superior qualitative and quantitative results. Our code is available at hrefhttps://github.com/rese1f/StableVideothis https URL.

Key facts. Cited by 182 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 44 of its 53 references that are in the map. First posted on arXiv 18 Aug 2023.

Builds on

44 of 53 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023Video Probabilistic Diffusion Models in Projected Latent Space15 Feb
  2. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  3. 2023Dreamix: Video Diffusion Models are General Video Editors2 Feb
  4. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
  5. 2022SinFusion: Training Diffusion Models on a Single Image or Video21 Nov
  6. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov
  7. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  8. 2022Video Diffusion Models7 Apr

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  10. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  12. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
24 more
  1. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  2. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  3. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  4. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  5. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  6. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  7. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  8. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  9. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  10. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  11. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  12. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  13. 2023Zero-shot Image-to-Image Translationcited by 503
  14. 2022MAGVIT: Masked Generative Video Transformercited by 341
  15. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  16. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  17. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  18. 2016Artistic style transfer for videoscited by 54
  19. 2020Layered Neural Rendering for Retiming People in Videocited by 51
  20. 2020Interactive Video Stylization Using Few-Shot Patch-Based Trainingcited by 43
  21. 2023DiffFashion: Reference-based Fashion Design with Structure-aware Transfer by Diffusion Modelscited by 33
  22. 2022Temporally Consistent Semantic Video Editingcited by 28
  23. 2022AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Resultscited by 20
  24. 2022Minimal Neural Atlas: Parameterizing Complex Surfaces with Minimal Charts and Distortioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 31; 2024: 85; 2025: 53; 2026: 13.

182 papers.

Sort:

By area:

  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  2. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  3. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  4. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 289
  5. 2023A Survey on Video Diffusion Modelscited by 203
  6. 2024Taming Rectified Flow for Inversion and Editingcited by 200
  7. 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 194
  8. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 186
  9. 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motioncited by 167
  10. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  11. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  12. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 124
38 more
  1. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 101
  2. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
  3. 2024SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memorycited by 91
  4. 2023VideoLCM: Video Latent Consistency Modelcited by 90
  5. 2024Animate-X: Universal Character Image Animation with Enhanced Motion Representationcited by 86
  6. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedbackcited by 85
  7. 2025Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuningcited by 81
  8. 2023CCEdit: Creative and Controllable Video Editing via Diffusion Modelscited by 80
  9. 2023FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesiscited by 79
  10. 2025Scaling Instruction-Based Video Editing with a High-Quality Synthetic Datasetcited by 76
  11. 2023VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondencecited by 69
  12. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  13. 2023MagicStick: Controllable Video Editing via Control Handle Transformationscited by 62
  14. 2025Follow-Your-Creation: Empowering 4D Creation through Video Inpaintingcited by 61
  15. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 60
  16. 2024PAD: Personalized Alignment of LLMs at Decoding-Timecited by 58
  17. 2023MotionEditor: Editing Video Motion via Content-Aware Diffusioncited by 56
  18. 2024Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Modelscited by 55
  19. 2024Follow-Your-Click: Open-domain Regional Image Animation via Short Promptscited by 52
  20. 2023A Recipe for Scaling up Text-to-Video Generation with Text-free Videoscited by 51
  21. 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editingcited by 50
  22. 2024Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editingcited by 49
  23. 2024Towards Understanding the Working Mechanism of Text-to-Image Diffusion Modelcited by 48
  24. 2024UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editingcited by 47
  25. 2024Learn "No" to Say "Yes" Better: Improving Vision-Language Models via Negationscited by 45
  26. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  27. 2023LivePhoto: Real Image Animation with Text-guided Motion Controlcited by 44
  28. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  29. 2024Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditionscited by 42
  30. 2024WorldGPT: Empowering LLM as Multimodal World Modelcited by 42
  31. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 42
  32. 2024Mora: Enabling Generalist Video Generation via A Multi-Agent Frameworkcited by 41
  33. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40
  34. 2023Fairy: Fast Parallelized Instruction-Guided Video-to-Video Synthesiscited by 39
  35. 2024ViViD: Video Virtual Try-on using Diffusion Modelscited by 38
  36. 2025DICEPTION: A Generalist Diffusion Model for Visual Perceptual Taskscited by 37
  37. 2024VideoDirector: Precise Video Editing via Text-to-Video Modelscited by 36
  38. 2025How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Modelscited by 31

50 most cited of 182 All 182 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editing
  2. 2023Pix2Video: Video Editing using Image Diffusion
  3. 2024COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editing
  4. 2025Efficient Temporal Consistency in Diffusion-Based Video Editing with Adaptor Modules: A Theoretical Framework
  5. 2023Edit Temporal-Consistent Videos with Image Diffusion Model
  6. 2024Slicedit: Zero-Shot Video Editing With Text-to-Image Diffusion Models Using Spatio-Temporal Slices
  7. 2026Vera: A Layered Diffusion Model for Content-Preserving Video Editing
  8. 2023Edit-A-Video: Single Video Editing with Object-Aware Consistency