HarmonPaint: Harmonized Training-Free Diffusion Inpainting
On the map: Diffusion models → 3D, point cloud and scene generation
Abstract (arXiv)
Existing inpainting methods often require extensive retraining or fine-tuning to integrate new content seamlessly, yet they struggle to maintain coherence in both structure and style between inpainted regions and the surrounding background. Motivated by these limitations, we introduce HarmonPaint, a training-free inpainting framework that seamlessly integrates with the attention mechanisms of diffusion models to achieve high-quality, harmonized image inpainting without any form of training. By leveraging masking strategies within self-attention, HarmonPaint ensures structural fidelity without model retraining or fine-tuning. Additionally, we exploit intrinsic diffusion model properties to transfer style information from unmasked to masked regions, achieving a harmonious integration of styles. Extensive experiments demonstrate the effectiveness of HarmonPaint across diverse scenes and styles, validating its versatility and performance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 63 of its 72 references that are in the map. First posted on arXiv 22 Jul 2025.
Builds on
63 of 72 references.
Closest ancestors · 6
Same direction, newest first
- 2024Attentive Eraser: Unleashing Diffusion Model's Object Removal Potential via Self-Attention Redirection Guidance17 Dec
- 2024Improving Text-guided Object Inpainting with Semantic Pre-inpainting12 Sep
- 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion11 Mar
- 2023Relightful Harmonization: Lighting-aware Portrait Background Replacement11 Dec
- 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model9 Dec
- 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan
Other references · 57
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
38 more
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022Generative Adversarial Networkscited by 5,627
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
- 2021Alias-Free Generative Adversarial Networkscited by 1,184
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
- 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
- 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 927
- 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
- 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
- 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
- 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
- 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
- 2022Blended Latent Diffusioncited by 432
- 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 310
- 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
- 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 233
- 2023Style Aligned Image Generation via Shared Attentioncited by 216
- 2017VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learningcited by 200
- 2023Generative Diffusion Prior for Unified Image Restoration and Enhancementcited by 199
- 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 196
- 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
- 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 169
- 2023Dense Text-to-Image Generation with Attention Modulationcited by 163
- 2019Pluralistic Image Completioncited by 162
- 2021Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAEcited by 106
- 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Modelscited by 75
- 2021Region-aware Adaptive Instance Normalization for Image Harmonizationcited by 71
- 2024Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Generationcited by 69
- 2023UDiffText: A Unified Framework for High-quality Text Synthesis in Arbitrary Images via Character-aware Diffusion Modelscited by 59
50 most cited of 57 All 57 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 20243D-Consistent Image Inpainting with Diffusion Models
- 2023Painterly Image Harmonization using Diffusion Model
- 2025IS-Diff: Improving Diffusion-Based Inpainting with Better Initial Seed
- 2025FreeInpaint: Tuning-free Prompt Alignment and Visual Rationality Enhancement in Image Inpainting
- 2023Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Model
- 2025Towards Seamless Borders: A Method for Mitigating Inconsistencies in Image Inpainting and Outpainting
- 2023Image Harmonization with Diffusion Model
- 2025GuidPaint: Class-Guided Image Inpainting with Diffusion Models