mlsanityData to 7 Oct 2026

Paint by Example: Exemplar-based Image Editing with Diffusion Models

arXiv 2211.13227 ↗First posted 23 Nov 2022cs.CVcited by 489

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Language-guided image editing has achieved great success recently. In this paper, for the first time, we investigate exemplar-guided image editing for more precise control. We achieve this goal by leveraging self-supervised training to disentangle and re-organize the source image and the exemplar. However, the naive approach will cause obvious fusing artifacts. We carefully analyze it and propose an information bottleneck and strong augmentations to avoid the trivial solution of directly copying and pasting the exemplar image. Meanwhile, to ensure the controllability of the editing process, we design an arbitrary shape mask for the exemplar image and leverage the classifier-free guidance to increase the similarity to the exemplar image. The whole framework involves a single forward of the diffusion model without any iterative optimization. We demonstrate that our method achieves an impressive performance and enables controllable editing on in-the-wild images with high fidelity.

Key facts. Cited by 489 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 60 of its 80 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

60 of 80 references.

Closest ancestors · 10

Same direction, newest first

  1. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  2. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  3. 2022Pretraining is All You Need for Image-to-Image Translation25 May
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  7. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidance10 Dec
  8. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov
  9. 2021Palette: Image-to-Image Diffusion Models10 Nov
  10. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation6 Oct

Other references · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2022Generative Adversarial Networkscited by 5,603
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  12. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
38 more
  1. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  2. 2016Colorful Image Colorizationcited by 1,113
  3. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  4. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  5. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  6. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  7. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  8. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  9. 2018Albumentations: fast and flexible image augmentationscited by 558
  10. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 542
  11. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  12. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  13. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  14. 2019FaceShifter: Towards High Fidelity And Occlusion Aware Face Swappingcited by 221
  15. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  16. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  17. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  18. 2021StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generatorscited by 164
  19. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  20. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  21. 2019DoveNet: Deep Image Harmonization via Domain Verificationcited by 134
  22. 2017Deep Image Harmonizationcited by 120
  23. 2020Cross-domain Correspondence Learning for Exemplar-based Image Translationcited by 116
  24. 2021Paint by Wordcited by 107
  25. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  26. 2019Deep Exemplar-based Video Colorizationcited by 75
  27. 2019Improving the Harmony of the Composite Image by Spatial-Separated Attention Modulecited by 73
  28. 2021Making Images Real Again: A Comprehensive Survey on Deep Image Compositioncited by 63
  29. 2022Improved Vector Quantized Diffusion Modelscited by 62
  30. 2019Mask-Guided Portrait Editing with Conditional GANscited by 61
  31. 2020GIQA: Generated Image Quality Assessmentcited by 51
  32. 2020Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applicationscited by 48
  33. 2022DCCF: Deep Comprehensible Color Filter Learning Framework for High-Resolution Image Harmonizationcited by 46
  34. 2017Aesthetic-Driven Image Enhancement by Adversarial Learningcited by 38
  35. 2020GuidedStyle: Attribute Knowledge Guided Style Manipulation for Semantic Face Editingcited by 37
  36. 2021High-Fidelity and Arbitrary Face Editingcited by 35
  37. 2020Semantic Pyramid for Image Generationcited by 34
  38. 2020Deep Image Compositingcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 97; 2024: 196; 2025: 126; 2026: 70.

489 papers.

Sort:

By area:

  1. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  2. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  3. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  4. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  5. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  6. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 297
  7. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 264
  8. 2023Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylizationcited by 251
  9. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 242
  10. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  11. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  12. 2023TextDiffuser: Diffusion Models as Text Painterscited by 223
38 more
  1. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  2. 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
  3. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  4. 2023StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-Oncited by 173
  5. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 165
  6. 2023InstructDiffusion: A Generalist Modeling Interface for Vision Taskscited by 162
  7. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 153
  8. 2024Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesiscited by 153
  9. 2023LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-Oncited by 151
  10. 2024OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-oncited by 150
  11. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 142
  12. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 132
  13. 2023Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flowcited by 123
  14. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
  15. 2023Self-correcting LLM-controlled Diffusion Modelscited by 112
  16. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 111
  17. 2024Improving Diffusion Models for Authentic Virtual Try-on in the Wildcited by 109
  18. 2024Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advancescited by 105
  19. 2023Cones 2: Customizable Image Synthesis with Multiple Subjectscited by 101
  20. 2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 100
  21. 2024CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Modelscited by 99
  22. 2024MotionBooth: Motion-Aware Customized Text-to-Video Generationcited by 92
  23. 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 85
  24. 2025DreamO: A Unified Framework for Image Customizationcited by 82
  25. 2023AVID: Any-Length Video Inpainting with Diffusion Modelcited by 81
  26. 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 79
  27. 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Modelscited by 75
  28. 2023TaleCrafter: Interactive Story Visualization with Multiple Characterscited by 71
  29. 2024ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertioncited by 71
  30. 2023Directed Diffusion: Direct Control of Object Placement through Attention Guidancecited by 69
  31. 2024IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representationcited by 64
  32. 2024Zero-shot Image Editing with Reference Imitationcited by 64
  33. 2023Inserting Anybody in Diffusion Models via Celeb Basiscited by 63
  34. 2023DiffusionLight: Light Probes for Free by Painting a Chrome Ballcited by 63
  35. 2023Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Modelscited by 62
  36. 2025Insert Anything: Image Insertion via In-Context Editing in DiTcited by 61
  37. 2023Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Modelcited by 60
  38. 2023ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulationcited by 59

50 most cited of 489 All 489 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Controllability-Fidelity Frontier in Diffusion Editing
  2. 2024Streamlining Image Editing with Layered Diffusion Brushes
  3. 2026Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offs
  4. 2023LayerDiffusion: Layered Controlled Image Editing with Diffusion Models
  5. 2026Diffusion Editing with Soft Mask: Pixel Level Redo of Image and Video with Adjustable Strength
  6. 2025CPAM: Context-Preserving Adaptive Manipulation for Zero-Shot Real Image Editing
  7. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions
  8. 2026SpecEdit: Training-Free Acceleration for Diffusion based Image Editing via Semantic Locking