mlsanityData to 7 Oct 2026

DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing

arXiv 2306.14435 ↗First posted 26 Jun 2023cs.CV cs.LGcited by 293

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Accurate and controllable image editing is a challenging task that has attracted significant attention recently. Notably, DragGAN is an interactive point-based image editing framework that achieves impressive editing results with pixel-level precision. However, due to its reliance on generative adversarial networks (GANs), its generality is limited by the capacity of pretrained GAN models. In this work, we extend this editing framework to diffusion models and propose a novel approach DragDiffusion. By harnessing large-scale pretrained diffusion models, we greatly enhance the applicability of interactive point-based editing on both real and diffusion-generated images. Our approach involves optimizing the diffusion latents to achieve precise spatial control. The supervision signal of this optimization process is from the diffusion model's UNet features, which are known to contain rich semantic and geometric information. Moreover, we introduce two additional techniques, namely LoRA fine-tuning and latent-MasaCtrl, to further preserve the identity of the original image. Lastly, we present a challenging benchmark dataset called DragBench -- the first benchmark to evaluate the performance of interactive point-based image editing methods. Experiments across a wide range of challenging cases (e.g., images with multiple objects, diverse object categories, various styles, etc.) demonstrate the versatility and generality of DragDiffusion. Code: https://github.com/Yujun-Shi/DragDiffusion.

Key facts. Cited by 293 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 46 of its 62 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

46 of 62 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023Emergent Correspondence from Image Diffusion6 Jun
  2. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
  3. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence24 May
  4. 2023Guided Image Synthesis via Initial Image Editing in Diffusion Model5 May
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  6. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov
  7. 2022MagicMix: Semantic Mixing with Diffusion Models28 Oct
  8. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May

Other references · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  9. 2020Denoising Diffusion Implicit Modelscited by 9,655
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
24 more
  1. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  3. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  4. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  5. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  6. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  7. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  8. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  9. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  10. 2023Zero-shot Image-to-Image Translationcited by 503
  11. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  12. 2023Universal Guidance for Diffusion Modelscited by 426
  13. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  14. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  15. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  16. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  17. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  18. 2023FreeDoM: Training-Free Energy-Guided Conditional Diffusion Modelcited by 290
  19. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 267
  20. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
  21. 2016Inverting The Generator Of A Generative Adversarial Networkcited by 140
  22. 2022User-Controllable Latent Transformer for StyleGAN Image Layout Editingcited by 28
  23. 2022Rewriting Geometric Rules of a GANcited by 25
  24. 2023LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesiscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 27; 2024: 99; 2025: 120; 2026: 47.

293 papers.

Sort:

By area:

  1. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
  2. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  3. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  4. 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 194
  5. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 193
  6. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  7. 2024In-Context LoRA for Diffusion Transformerscited by 155
  8. 2023Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Codecited by 145
  9. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 132
  10. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  11. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 126
  12. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
38 more
  1. 2024GaussCtrl: Multi-View Consistent Text-Driven 3D Gaussian Splatting Editingcited by 114
  2. 2024Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advancescited by 105
  3. 2023Diffusion Model as Representation Learnercited by 95
  4. 2024DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editingcited by 92
  5. 2024DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image Editingcited by 88
  6. 2024ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generationcited by 74
  7. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  8. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  9. 2024Not All Noises Are Created Equally:Diffusion Noise Selection and Optimizationcited by 70
  10. 2023VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondencecited by 69
  11. 2024SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generationcited by 68
  12. 2023The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editingcited by 61
  13. 2024Video Diffusion Models are Training-free Motion Interpreter and Controllercited by 60
  14. 2023Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraintscited by 59
  15. 2023MotionEditor: Editing Video Motion via Content-Aware Diffusioncited by 56
  16. 2024LightIt: Illumination Modeling and Control for Diffusion Modelscited by 56
  17. 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformercited by 56
  18. 2023Readout Guidance: Learning Control from Diffusion Featurescited by 52
  19. 2024MagicTime: Time-lapse Video Generation Models as Metamorphic Simulatorscited by 51
  20. 2024Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagationcited by 50
  21. 2023Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Modelscited by 48
  22. 2024Good Seed Makes a Good Crop: Discovering Secret Seeds in Text-to-Image Diffusion Modelscited by 47
  23. 2025Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Modelcited by 46
  24. 2024Image Sculpting: Precise Object Editing with 3D Geometry Controlcited by 45
  25. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  26. 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
  27. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  28. 2023Exact Diffusion Inversion via Bi-directional Integration Approximationcited by 43
  29. 2023MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learningcited by 42
  30. 2024Wear-Any-Way: Manipulable Virtual Try-on via Sparse Correspondence Alignmentcited by 42
  31. 2023DragVideo: Interactive Drag-style Video Editingcited by 40
  32. 2024Motion Guidance: Diffusion-Based Image Editing with Differentiable Motion Estimatorscited by 39
  33. 2024Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videoscited by 38
  34. 2023Exploiting Diffusion Prior for Generalizable Dense Predictioncited by 37
  35. 2024EditWorld: Simulating World Dynamics for Instruction-Following Image Editingcited by 37
  36. 2024VideoDirector: Precise Video Editing via Text-to-Video Modelscited by 36
  37. 2024MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesiscited by 35
  38. 2024GeoDiffuser: Geometry-Based Image Editing with Diffusion Modelscited by 34

50 most cited of 293 All 293 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Enabling Local Editing in Diffusion Models by Joint and Individual Component Analysis
  2. 2022Fine-grained Image Editing by Pixel-wise Guidance Using Diffusion Models
  3. 2022Blended Latent Diffusion
  4. 2023Diffusion Brush: A Latent Diffusion Model-based Editing Tool for AI-generated Images
  5. 2025Beyond Sliders: Mastering the Art of Diffusion-based Image Manipulation
  6. 2024E2GAN: Efficient Training of Efficient GANs for Image-to-Image Translation
  7. 2023Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D
  8. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation