# HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image Inpainting with Diffusion Models

arXiv: https://arxiv.org/abs/2312.14091 · HTML: https://mlsanity.com/paper/2312.14091/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 41 of its 52 references that are in the map.
- First posted on arXiv 21 Dec 2023.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 41)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)

## Built on by (most cited first, 30 of 67)

- [Diffusion Model-Based Image Editing: A Survey](https://mlsanity.com/paper/2402.17525/) (2024)
- [BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion](https://mlsanity.com/paper/2403.06976/) (2024)
- [Generative Image Layer Decomposition with Visual Effects](https://mlsanity.com/paper/2411.17864/) (2024)
- [BrushEdit: All-In-One Image Inpainting and Editing](https://mlsanity.com/paper/2412.10316/) (2024)
- [DreamOmni: Unified Image Generation and Editing](https://mlsanity.com/paper/2412.17098/) (2024)
- [Follow-Your-Preference: Towards Preference-Aligned Image Inpainting](https://mlsanity.com/paper/2509.23082/) (2025)
- [PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference](https://mlsanity.com/paper/2410.21966/) (2024)
- [Reflecting Reality: Enabling Diffusion Models to Produce Faithful Mirror Reflections](https://mlsanity.com/paper/2409.14677/) (2024)
- [TurboFill: Adapting Few-step Text-to-image Model for Fast Image Inpainting](https://mlsanity.com/paper/2504.00996/) (2025)
- [MTV-Inpaint: Multi-Task Long Video Inpainting](https://mlsanity.com/paper/2503.11412/) (2025)
- [AdvPaint: Protecting Images from Inpainting Manipulation via Adversarial Attention Disruption](https://mlsanity.com/paper/2503.10081/) (2025)
- [Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis](https://mlsanity.com/paper/2406.04032/) (2024)
- [CorrFill: Enhancing Faithfulness in Reference-based Inpainting with Correspondence Guidance in Diffusion Models](https://mlsanity.com/paper/2501.02355/) (2025)
- [MirrorVerse: Pushing Diffusion Models to Realistically Reflect the World](https://mlsanity.com/paper/2504.15397/) (2025)
- [Object Isolated Attention for Consistent Story Visualization](https://mlsanity.com/paper/2503.23353/) (2025)
- [Anywhere: A Multi-Agent Framework for User-Guided, Reliable, and Diverse Foreground-Conditioned Image Generation](https://mlsanity.com/paper/2404.18598/) (2024)
- [SOEDiff: Efficient Distillation for Small Object Editing](https://mlsanity.com/paper/2405.09114/) (2024)
- [StdGEN: Semantic-Decomposed 3D Character Generation from Single Images](https://mlsanity.com/paper/2411.05738/) (2024)
- [Improving Editability in Image Generation with Layer-wise Memory](https://mlsanity.com/paper/2505.01079/) (2025)
- [Trans-Adapter: A Plug-and-Play Framework for Transparent Image Inpainting](https://mlsanity.com/paper/2508.01098/) (2025)
- [ReTrack: Data Unlearning in Diffusion Models through Redirecting the Denoising Trajectory](https://mlsanity.com/paper/2509.13007/) (2025)
- [Test-Time Anchoring for Discrete Diffusion Posterior Sampling](https://mlsanity.com/paper/2510.02291/) (2025)
- [One Stone with Two Birds: A Null-Text-Null Frequency-Aware Diffusion Models for Text-Guided Image Inpainting](https://mlsanity.com/paper/2510.08273/) (2025)
- [HiFi-Inpaint: Towards High-Fidelity Reference-Based Inpainting for Generating Detail-Preserving Human-Product Images](https://mlsanity.com/paper/2603.02210/) (2026)
- [DA-HFNet: Progressive Fine-Grained Forgery Image Detection and Localization Based on Dual Attention](https://mlsanity.com/paper/2406.01489/) (2024)
- [FreeCond: Free Lunch in the Input Conditions of Text-Guided Inpainting](https://mlsanity.com/paper/2412.00427/) (2024)
- [OutDreamer: Video Outpainting with a Diffusion Transformer](https://mlsanity.com/paper/2506.22298/) (2025)
- [SONIC: Spectral Optimization of Noise for Inpainting with Consistency](https://mlsanity.com/paper/2511.19985/) (2025)
- [Streamlining Image Editing with Layered Diffusion Brushes](https://mlsanity.com/paper/2405.00313/) (2024)
- [AnyLogo: Symbiotic Subject-Driven Diffusion System with Gemini Status](https://mlsanity.com/paper/2409.17740/) (2024)

## Abstract (arXiv)

Recent progress in text-guided image inpainting, based on the unprecedented success of text-to-image diffusion models, has led to exceptionally realistic and visually plausible results. However, there is still significant potential for improvement in current text-to-image inpainting models, particularly in better aligning the inpainted area with user prompts and performing high-resolution inpainting. Therefore, we introduce HD-Painter, a training free approach that accurately follows prompts and coherently scales to high resolution image inpainting. To this end, we design the Prompt-Aware Introverted Attention (PAIntA) layer enhancing self-attention scores by prompt information resulting in better text aligned generations. To further improve the prompt coherence we introduce the Reweighting Attention Score Guidance (RASG) mechanism seamlessly integrating a post-hoc sampling strategy into the general form of DDIM to prevent out-of-distribution latent shifts. Moreover, HD-Painter allows extension to larger scales by introducing a specialized super-resolution technique customized for inpainting, enabling the completion of missing regions in images of up to 2K resolution. Our experiments demonstrate that HD-Painter surpasses existing state-of-the-art approaches quantitatively and qualitatively across multiple metrics and a user study. Code is publicly available at: https://github.com/Picsart-AI-Research/HD-Painter