# RAD: Region-Aware Diffusion Models for Image Inpainting

arXiv: https://arxiv.org/abs/2412.09191 · HTML: https://mlsanity.com/paper/2412.09191/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 43 of its 56 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 43)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model](https://mlsanity.com/paper/2212.05034/) (2022)
- [Diffusion Probabilistic Modeling for Video Generation](https://mlsanity.com/paper/2203.09481/) (2022)

## Built on by (most cited first, 20 of 20)

- [Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising](https://mlsanity.com/paper/2511.08633/) (2025)
- [ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models](https://mlsanity.com/paper/2603.00492/) (2026)
- [MGE-LDM: Joint Latent Diffusion for Simultaneous Music Generation and Source Extraction](https://mlsanity.com/paper/2505.23305/) (2025)
- [SONIC: Spectral Optimization of Noise for Inpainting with Consistency](https://mlsanity.com/paper/2511.19985/) (2025)
- [GuidPaint: Class-Guided Image Inpainting with Diffusion Models](https://mlsanity.com/paper/2507.21627/) (2025)
- [SimLBR: Learning to Detect Fake Images by Learning to Detect Real Images](https://mlsanity.com/paper/2602.20412/) (2026)
- [NeoMap: Training-free Novel-View Synthesis from Single Images and Videos](https://mlsanity.com/paper/2607.01962/) (2026)
- [Transformer-Based Inpainting for Real-Time 3D Streaming in Sparse Multi-Camera Setups](https://mlsanity.com/paper/2603.05507/) (2026)
- [Longitudinal Lesion Inpainting in Brain MRI via 3D Region Aware Diffusion](https://mlsanity.com/paper/2603.05693/) (2026)
- [Enhancing Image Aesthetics with Dual-Conditioned Diffusion Models Guided by Multimodal Perception](https://mlsanity.com/paper/2603.11556/) (2026)
- [Follow-Your-Preference++: Rethinking Preference Alignment for Image Inpainting](https://mlsanity.com/paper/2606.03216/) (2026)
- [RQUL-UIE: Revitalizing Quality-Unstable Labels for Underwater Image Enhancement via In-Dataset Self-Supervision](https://mlsanity.com/paper/2606.06176/) (2026)
- [AsyncPatch Diffusion: spatially-flexible image generation](https://mlsanity.com/paper/2606.07079/) (2026)
- [Fusing Transferred Priors and Physics-based Decomposition for Underwater Image Enhancement](https://mlsanity.com/paper/2606.15648/) (2026)
- [Inpainting Insights: Elevating Visual XAI with Photorealistic Perturbations](https://mlsanity.com/paper/2607.15482/) (2026)
- [Physics-aware Masked Diffusion-based Flood Simulation for Urban Fisheye Disaster Detection](https://mlsanity.com/paper/2607.15527/) (2026)
- [When Diffusion Models Forget Who You Are: Identity Preservation in Face Inpainting under Large Occlusions](https://mlsanity.com/paper/2608.04820/) (2026)
- [SUMI: Scalable Unified Model for 3D Point Cloud Inference](https://mlsanity.com/paper/2608.08115/) (2026)
- [RARF: Region-Aware Rectified Flows for 3D Brain MRI Inpainting](https://mlsanity.com/paper/2609.03956/) (2026)
- [NOFACE3D: NOise-scheduled Focused Anchoring and Controlled Enhancement for 3D Generation](https://mlsanity.com/paper/2610.13050/) (2026)

## Abstract (arXiv)

Diffusion models have achieved remarkable success in image generation, with applications broadening across various domains. Inpainting is one such application that can benefit significantly from diffusion models. Existing methods either hijack the reverse process of a pretrained diffusion model or cast the problem into a larger framework, ie, conditioned generation. However, these approaches often require nested loops in the generation process or additional components for conditioning. In this paper, we present region-aware diffusion models (RAD) for inpainting with a simple yet effective reformulation of the vanilla diffusion models. RAD utilizes a different noise schedule for each pixel, which allows local regions to be generated asynchronously while considering the global image context. A plain reverse process requires no additional components, enabling RAD to achieve inference time up to 100 times faster than the state-of-the-art approaches. Moreover, we employ low-rank adaptation (LoRA) to fine-tune RAD based on other pretrained diffusion models, reducing computational burdens in training as well. Experiments demonstrated that RAD provides state-of-the-art results both qualitatively and quantitatively, on the FFHQ, LSUN Bedroom, and ImageNet datasets.