mlsanityData to 7 Oct 2026

Diffusion Model-Based Image Editing: A Survey

arXiv 2402.17525 ↗First posted 27 Feb 2024cs.CVcited by 212

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Denoising diffusion models have emerged as a powerful tool for various image generation and editing tasks, facilitating the synthesis of visual content in an unconditional or input-conditional manner. The core idea behind them is learning to reverse the process of gradually adding noise to images, allowing them to generate high-quality samples from a complex distribution. In this survey, we provide an exhaustive overview of existing methods using diffusion models for image editing, covering both theoretical and practical aspects in the field. We delve into a thorough analysis and categorization of these works from multiple perspectives, including learning strategies, user-input conditions, and the array of specific editing tasks that can be accomplished. In addition, we pay special attention to image inpainting and outpainting, and explore both earlier traditional context-driven and current multimodal conditional methods, offering a comprehensive analysis of their methodologies. To further evaluate the performance of text-guided image editing algorithms, we propose a systematic benchmark, EditEval, featuring an innovative metric, LMM Score. Finally, we address current limitations and envision some potential directions for future research. The accompanying repository is released at https://github.com/SiatMMLab/Awesome-Diffusion-Model-Based-Image-Editing-Methods.

Key facts. Cited by 212 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 287 of its 329 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

287 of 329 references.

Closest ancestors · 94

Same direction, newest first

  1. 2023ZONE: Zero-Shot Instruction-Guided Local Editing28 Dec
  2. 2023Tuning-Free Inversion-Enhanced Control for Consistent Image Editing22 Dec
  3. 2023SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models11 Dec
  4. 2023Personalized Face Inpainting with Diffusion Models by Parallel Visual Attention6 Dec
  5. 2023LEDITS++: Limitless Image Editing using Text-to-Image Models28 Nov
  6. 2023The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing2 Nov
  7. 2023CustomNet: Zero-shot Object Customization with Variable-Viewpoints in Text-to-Image Diffusion Models30 Oct
  8. 2023Learning to Follow Object-Centric Image Editing Instructions Faithfully29 Oct
  9. 2023CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulation19 Oct
  10. 2023EditVal: Benchmarking Diffusion Based Text-Guided Image Editing Methods3 Oct
  11. 2023Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editing27 Sep
  12. 2023FEC: Three Finetuning-free Methods to Enhance Consistency for Real Image Editing26 Sep
38 more
  1. 2023Face Aging via Diffusion-based Editing20 Sep
  2. 2023Forgedit: Text Guided Image Editing via Learning and Forgetting19 Sep
  3. 2023Effective Real Image Editing with Accelerated Iterative Diffusion Inversion10 Sep
  4. 2023MoEController: Instruction-based Arbitrary Image Manipulation with Mixture-of-Expert Controllers8 Sep
  5. 2023Iterative Multi-granular Image Editing using Diffusion Models1 Sep
  6. 2023A Survey of Diffusion Based Image Generation Models: Issues and Their Solutions25 Aug
  7. 2023Watch Your Steps: Local Image and Scene Editing by Text Instructions17 Aug
  8. 2023StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models15 Aug
  9. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  10. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition24 Jul
  11. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models5 Jul
  12. 2023LEDITS: Real Image Editing with DDPM Inversion and Semantic Guidance2 Jul
  13. 2023PFB-Diff: Progressive Feature Blending Diffusion for Text-driven Image Editing28 Jun
  14. 2023DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models26 Jun
  15. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing26 Jun
  16. 2023Energy-Based Cross Attention for Bayesian Context Update in Text-to-Image Diffusion Models16 Jun
  17. 2023Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Model13 Jun
  18. 2023Improving Tuning-Free Real Image Editing with Proximal Guidance8 Jun
  19. 2023User-friendly Image Editing with Minimal Text Input: Leveraging Captioning and Injection Techniques5 Jun
  20. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds1 Jun
  21. 2023Differential Diffusion: Giving Each Pixel Its Strength1 Jun
  22. 2023FaceDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Models1 Jun
  23. 2023Diffusion Brush: A Latent Diffusion Model-based Editing Tool for AI-generated Images31 May
  24. 2023LayerDiffusion: Layered Controlled Image Editing with Diffusion Models30 May
  25. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
  26. 2023InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructions29 May
  27. 2023Conditional Score Guidance for Text-Driven Image-to-Image Translation29 May
  28. 2023Text-to-image Editing by Image Information Removal27 May
  29. 2023Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference27 May
  30. 2023Negative-prompt Inversion: Fast Image Inversion for Editing with Text-guided Diffusion Models26 May
  31. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models25 May
  32. 2023Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models25 May
  33. 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Models25 May
  34. 2023ChatFace: Chat-Guided Real Face Editing via Diffusion Latent Space Manipulation24 May
  35. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild18 May
  36. 2023iEdit: Localised Text-guided Image Editing with Weak Supervision10 May
  37. 2023ReGeneration Learning of Diffusion Models with Rich Prompts for Zero-Shot Image Translation8 May
  38. 2023Prompt Tuning Inversion for Text-Driven Image Editing Using Diffusion Models8 May

50 newest of 94 All 94 (JSON)

Other references · 193

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  10. 2020Denoising Diffusion Implicit Modelscited by 9,655
  11. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  12. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
38 more
  1. 2023Segment Anythingcited by 8,158
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  8. 2017Neural Discrete Representation Learningcited by 4,650
  9. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  11. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  13. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  14. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  15. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  16. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  17. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  18. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  19. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  20. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  21. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  22. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  23. 2022Video Diffusion Modelscited by 2,280
  24. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  25. 2015Variational Inference with Normalizing Flowscited by 2,161
  26. 2014Conditional Generative Adversarial Netscited by 2,152
  27. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  28. 2023Consistency Modelscited by 1,778
  29. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  30. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  31. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  32. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  33. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  34. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  35. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  36. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  37. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  38. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155

50 most cited of 193 All 193 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 58; 2025: 92; 2026: 62.

212 papers.

Sort:

By area:

  1. 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 200
  2. 2024Taming Rectified Flow for Inversion and Editingcited by 200
  3. 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 191
  4. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 111
  5. 2024Neural Gaffer: Relighting Any Object via Diffusioncited by 91
  6. 2025Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedbackcited by 78
  7. 2024Diffusion Models in Low-Level Vision: A Surveycited by 73
  8. 2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 69
  9. 2024I2EBench: A Comprehensive Benchmark for Instruction-based Image Editingcited by 61
  10. 2024TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negativescited by 57
  11. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  12. 2025K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAscited by 48
38 more
  1. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  2. 2024Steering Rectified Flow Models in the Vector Field for Controlled Image Generationcited by 38
  3. 2024EditWorld: Simulating World Dynamics for Instruction-Following Image Editingcited by 37
  4. 2024Learning Action and Reasoning-Centric Image Editing from Videos and Simulationscited by 32
  5. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  6. 2024BrushEdit: All-In-One Image Inpainting and Editingcited by 31
  7. 2024MagicQuill: An Intelligent Interactive Image Editing Systemcited by 30
  8. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  9. 2024ColorPeel: Color Prompt Learning with Diffusion Models via Color and Shape Disentanglementcited by 28
  10. 2024Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectivescited by 27
  11. 2024Pathways on the Image Manifold: Image Editing via Video Generationcited by 27
  12. 2025Follow-Your-Shape: Shape-Aware Image Editing via Trajectory-Guided Region Controlcited by 27
  13. 2024VideoAgent: Self-Improving Video Generationcited by 24
  14. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  15. 2025Edit Transfer: Learning Image Editing via Vision In-Context Relationscited by 23
  16. 2024Benchmarking Counterfactual Image Generationcited by 22
  17. 2025Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Designcited by 21
  18. 2025h-Edit: Effective and Flexible Diffusion-Based Editing via Doob's h-Transformcited by 18
  19. 2025InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignmentcited by 18
  20. 2024Diffusion Model for Planning: A Systematic Literature Reviewcited by 17
  21. 2024InsightEdit: Towards Better Instruction Following for Image Editingcited by 17
  22. 2024DIVE: Taming DINO for Subject-Driven Video Editingcited by 17
  23. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  24. 2025VINCIE: Unlocking In-context Image Editing from Videocited by 17
  25. 2024FreeCompose: Generic Zero-Shot Image Composition with Diffusion Priorcited by 16
  26. 2024MagicEraser: Erasing Any Objects via Semantics-Aware Controlcited by 15
  27. 2024MagicTailor: Component-Controllable Personalization in Text-to-Image Diffusion Modelscited by 14
  28. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  29. 2025Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferencescited by 13
  30. 2024ReMOVE: A Reference-free Metric for Object Erasurecited by 12
  31. 2025Get In Video: Add Anything You Want to the Videocited by 12
  32. 2025EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editingcited by 12
  33. 2026How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editingcited by 12
  34. 2024InstructGIE: Towards Generalizable Image Editingcited by 11
  35. 2024Conditioning diffusion models by explicit forward-backward bridgingcited by 11
  36. 2024Enhancing MMDiT-Based Text-to-Image Models for Similar Subject Generationcited by 11
  37. 2025Hallucination at a Glance: Controlled Visual Edits and Fine-Grained Multimodal Learningcited by 11
  38. 2026Group Editing: Edit Multiple Images in One Gocited by 11

50 most cited of 212 All 212 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Pix2Video: Video Editing using Image Diffusion
  2. 2023Guided Image Synthesis via Initial Image Editing in Diffusion Model
  3. 2023High-Fidelity Diffusion-based Image Editing
  4. 2024Paint by Inpaint: Learning to Add Image Objects by Removing Them First
  5. 2022On Conditioning the Input Noise for Controlled Image Generation with Diffusion Models
  6. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  7. 2024Diffusion-based image inpainting with internal learning
  8. 2024A Comprehensive Survey on Diffusion Models and Their Applications