mlsanityData to 7 Oct 2026

SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations

arXiv 2108.01073 ↗First posted 2 Aug 2021cs.CV cs.AIcited by 1,903

Not on the map. Why?

Cited by 1,134 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Guided image synthesis enables everyday users to create and edit photo-realistic images with minimum effort. The key challenge is balancing faithfulness to the user input (e.g., hand-drawn colored strokes) and realism of the synthesized image. Existing GAN-based methods attempt to achieve such balance using either conditional GANs or GAN inversions, which are challenging and often require additional training data or loss functions for individual applications. To address these issues, we introduce a new image synthesis and editing method, Stochastic Differential Editing (SDEdit), based on a diffusion model generative prior, which synthesizes realistic images by iteratively denoising through a stochastic differential equation (SDE). Given an input image with user guide of any type, SDEdit first adds noise to the input, then subsequently denoises the resulting image through the SDE prior to increase its realism. SDEdit does not require task-specific training or inversions and can naturally achieve the balance between realism and faithfulness. SDEdit significantly outperforms state-of-the-art GAN-based methods by up to 98.09% on realism and 91.72% on overall satisfaction scores, according to a human perception study, on multiple tasks, including stroke-based image synthesis and editing as well as image compositing.

Key facts. Cited by 1,903 papers in the map (data through 7 Oct 2026). Builds on 59 of its 74 references that are in the map. First posted on arXiv 2 Aug 2021.

Builds on

59 of 74 references.

References · 59

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  11. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  12. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
38 more
  1. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  2. 2018Demystifying MMD GANscited by 1,268
  3. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  4. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  5. 2016Colorful Image Colorizationcited by 1,113
  6. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  7. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  8. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  9. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  10. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  11. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  12. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  13. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  14. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  15. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  16. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  17. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  18. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 331
  19. 2018Context Encoding for Semantic Segmentationcited by 289
  20. 2021How to Train Your Energy-Based Modelscited by 271
  21. 2019SEAN: Image Synthesis with Semantic Region-Adaptive Normalizationcited by 253
  22. 2016Learning Representations for Automatic Colorizationcited by 251
  23. 2019On the "steerability" of generative adversarial networkscited by 247
  24. 2020Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulationcited by 225
  25. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  26. 2019GANalyze: Toward Visual Definitions of Cognitive Image Propertiescited by 193
  27. 2021SNIPS: Solving Noisy Inverse Problems Stochasticallycited by 191
  28. 2019Image Processing Using Multi-Code GAN Priorcited by 185
  29. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  30. 2016A Theory of Generative ConvNetcited by 174
  31. 2019Seeing What a GAN Cannot Generatecited by 148
  32. 2017Real-Time User-Guided Image Colorization with Learned Deep Priorscited by 123
  33. 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
  34. 2019SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Colorcited by 105
  35. 2020Rewriting a Deep Generative Modelcited by 105
  36. 2020Adversarial score matching and improved sampling for image generationcited by 95
  37. 2016High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesiscited by 89
  38. 2020Generative Hierarchical Features from Synthesizing Imagescited by 83

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 75; 2023: 408; 2024: 592; 2025: 481; 2026: 345.

1,903 papers.

Sort:

By area:

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  3. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  4. 2023Consistency Modelscited by 1,778
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  6. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  7. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  8. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  9. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  10. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  11. 2022Diffusion Models in Vision: A Surveycited by 1,008
  12. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
38 more
  1. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  2. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  3. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  4. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  5. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  6. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  7. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 707
  8. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  9. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  10. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  11. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  12. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  13. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  14. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  15. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  16. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  17. 2025VACE: All-in-One Video Creation and Editingcited by 467
  18. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  19. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  20. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  21. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  22. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  23. 2023Emergent Correspondence from Image Diffusioncited by 437
  24. 2022Blended Latent Diffusioncited by 430
  25. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  26. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  27. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  28. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  29. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 365
  30. 2023ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shiftingcited by 360
  31. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 358
  32. 2023Human Motion Diffusion as a Generative Priorcited by 354
  33. 2023DeepCache: Accelerating Diffusion Models for Freecited by 346
  34. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
  35. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  36. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 323
  37. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  38. 2023Pix2Video: Video Editing using Image Diffusioncited by 318

50 most cited of 1,903 All 1,903 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Effective Real Image Editing with Accelerated Iterative Diffusion Inversion
  2. 2023Guided Image Synthesis via Initial Image Editing in Diffusion Model
  3. 2022Semantic Image Synthesis via Diffusion Models
  4. 2025d-Sketch: Improving Visual Fidelity of Sketch-to-Image Translation with Pretrained Latent Diffusion Models without Retraining
  5. 2024Diffusion Model-Based Image Editing: A Survey
  6. 2023Filter-Guided Diffusion for Controllable Image Generation
  7. 2024E2GAN: Efficient Training of Efficient GANs for Image-to-Image Translation
  8. 2025Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceivers