# InpDiffusion: Image Inpainting Localization via Conditional Diffusion Models

arXiv: https://arxiv.org/abs/2501.02816 · HTML: https://mlsanity.com/paper/2501.02816/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 6 Jan 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 24 of 24)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction](https://mlsanity.com/paper/2112.05146/) (2021)
- [Deblurring via Stochastic Refinement](https://mlsanity.com/paper/2112.02475/) (2021)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning](https://mlsanity.com/paper/1901.00212/) (2019)
- [MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection](https://mlsanity.com/paper/2112.08935/) (2021)
- [EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protection](https://mlsanity.com/paper/2312.08883/) (2023)
- [Segmenting Transparent Objects in the Wild](https://mlsanity.com/paper/2003.13948/) (2020)
- [Aggregated Contextual Transformations for High-Resolution Image Inpainting](https://mlsanity.com/paper/2104.01431/) (2021)
- [Shift-Net: Image Inpainting via Deep Feature Rearrangement](https://mlsanity.com/paper/1801.09392/) (2018)
- [Region Normalization for Image Inpainting](https://mlsanity.com/paper/1911.10375/) (2019)
- [AutoSplice: A Text-prompt Manipulated Image Dataset for Media Forensics](https://mlsanity.com/paper/2304.06870/) (2023)
- [Progressive Deblurring of Diffusion Models for Coarse-to-Fine Image Synthesis](https://mlsanity.com/paper/2207.11192/) (2022)
- [Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention](https://mlsanity.com/paper/2009.01031/) (2020)
- [CFL-Net: Image Forgery Localization Using Contrastive Learning](https://mlsanity.com/paper/2210.02182/) (2022)
- [S3RP: Self-Supervised Super-Resolution and Prediction for Advection-Diffusion Process](https://mlsanity.com/paper/2111.04639/) (2021)

## Built on by (most cited first, 3 of 3)

- [PromptForge-350k: A Large-Scale Dataset and Contrastive Framework for Prompt-Based AI Image Forgery Localization](https://mlsanity.com/paper/2603.29386/) (2026)
- [Progressive Decision-Making for Localizing Open-Ended AI-Generated Image Forgeries](https://mlsanity.com/paper/2607.29156/) (2026)
- [SUMI: Scalable Unified Model for 3D Point Cloud Inference](https://mlsanity.com/paper/2608.08115/) (2026)

## Abstract (arXiv)

As artificial intelligence advances rapidly, particularly with the advent of GANs and diffusion models, the accuracy of Image Inpainting Localization (IIL) has become increasingly challenging. Current IIL methods face two main challenges: a tendency towards overconfidence, leading to incorrect predictions; and difficulty in detecting subtle tampering boundaries in inpainted images. In response, we propose a new paradigm that treats IIL as a conditional mask generation task utilizing diffusion models. Our method, InpDiffusion, utilizes the denoising process enhanced by the integration of image semantic conditions to progressively refine predictions. During denoising, we employ edge conditions and introduce a novel edge supervision strategy to enhance the model's perception of edge details in inpainted objects. Balancing the diffusion model's stochastic sampling with edge supervision of tampered image regions mitigates the risk of incorrect predictions from overconfidence and prevents the loss of subtle boundaries that can result from overly stochastic processes. Furthermore, we propose an innovative Dual-stream Multi-scale Feature Extractor (DMFE) for extracting multi-scale features, enhancing feature representation by considering both semantic and edge conditions of the inpainted images. Extensive experiments across challenging datasets demonstrate that the InpDiffusion significantly outperforms existing state-of-the-art methods in IIL tasks, while also showcasing excellent generalization capabilities and robustness.