mlsanityData to 14 Oct 2026

HarmonPaint: Harmonized Training-Free Diffusion Inpainting

arXiv 2507.16732 ↗First posted 22 Jul 2025cs.CVcited by 1

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Existing inpainting methods often require extensive retraining or fine-tuning to integrate new content seamlessly, yet they struggle to maintain coherence in both structure and style between inpainted regions and the surrounding background. Motivated by these limitations, we introduce HarmonPaint, a training-free inpainting framework that seamlessly integrates with the attention mechanisms of diffusion models to achieve high-quality, harmonized image inpainting without any form of training. By leveraging masking strategies within self-attention, HarmonPaint ensures structural fidelity without model retraining or fine-tuning. Additionally, we exploit intrinsic diffusion model properties to transfer style information from unmasked to masked regions, achieving a harmonious integration of styles. Extensive experiments demonstrate the effectiveness of HarmonPaint across diverse scenes and styles, validating its versatility and performance.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 63 of its 72 references that are in the map. First posted on arXiv 22 Jul 2025.

Builds on

63 of 72 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Attentive Eraser: Unleashing Diffusion Model's Object Removal Potential via Self-Attention Redirection Guidance17 Dec
  2. 2024Improving Text-guided Object Inpainting with Semantic Pre-inpainting12 Sep
  3. 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion11 Mar
  4. 2023Relightful Harmonization: Lighting-aware Portrait Background Replacement11 Dec
  5. 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model9 Dec
  6. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan

Other references · 57

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  8. 2020Denoising Diffusion Implicit Modelscited by 9,781
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
38 more
  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2022Classifier-Free Diffusion Guidancecited by 5,847
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  9. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  12. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  13. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  14. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  15. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  16. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 927
  17. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  18. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  19. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  20. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  21. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  22. 2022Blended Latent Diffusioncited by 432
  23. 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 310
  24. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  25. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 233
  26. 2023Style Aligned Image Generation via Shared Attentioncited by 216
  27. 2017VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learningcited by 200
  28. 2023Generative Diffusion Prior for Unified Image Restoration and Enhancementcited by 199
  29. 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 196
  30. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  31. 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 169
  32. 2023Dense Text-to-Image Generation with Attention Modulationcited by 163
  33. 2019Pluralistic Image Completioncited by 162
  34. 2021Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAEcited by 106
  35. 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Modelscited by 75
  36. 2021Region-aware Adaptive Instance Normalization for Image Harmonizationcited by 71
  37. 2024Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Generationcited by 69
  38. 2023UDiffText: A Unified Framework for High-quality Text Synthesis in Arbitrary Images via Character-aware Diffusion Modelscited by 59

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 20243D-Consistent Image Inpainting with Diffusion Models
  2. 2023Painterly Image Harmonization using Diffusion Model
  3. 2025IS-Diff: Improving Diffusion-Based Inpainting with Better Initial Seed
  4. 2025FreeInpaint: Tuning-free Prompt Alignment and Visual Rationality Enhancement in Image Inpainting
  5. 2023Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Model
  6. 2025Towards Seamless Borders: A Method for Mitigating Inconsistencies in Image Inpainting and Outpainting
  7. 2023Image Harmonization with Diffusion Model
  8. 2025GuidPaint: Class-Guided Image Inpainting with Diffusion Models