# Region-Aware Diffusion for Zero-shot Text-driven Image Editing

arXiv: https://arxiv.org/abs/2302.11797 · HTML: https://mlsanity.com/paper/2302.11797/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 41 of its 56 references that are in the map.
- First posted on arXiv 23 Feb 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 41)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [Text2LIVE: Text-Driven Layered Image and Video Editing](https://mlsanity.com/paper/2204.02491/) (2022)
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [CLIPstyler: Image Style Transfer with a Single Text Condition](https://mlsanity.com/paper/2112.00374/) (2021)
- [ManiGAN: Text-Guided Image Manipulation](https://mlsanity.com/paper/1912.06203/) (2019)

## Built on by (most cited first, 25 of 25)

- [Diffusion Model-Based Image Editing: A Survey](https://mlsanity.com/paper/2402.17525/) (2024)
- [Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling](https://mlsanity.com/paper/2401.15977/) (2024)
- [Taming Rectified Flow for Inversion and Editing](https://mlsanity.com/paper/2411.04746/) (2024)
- [Stable Flow: Vital Layers for Training-Free Image Editing](https://mlsanity.com/paper/2411.14430/) (2024)
- [ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models](https://mlsanity.com/paper/2305.16225/) (2023)
- [ControlVAR: Exploring Controllable Visual Autoregressive Modeling](https://mlsanity.com/paper/2406.09750/) (2024)
- [Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesis](https://mlsanity.com/paper/2304.03869/) (2023)
- [Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Models](https://mlsanity.com/paper/2508.12880/) (2025)
- [Conditional Image Synthesis with Diffusion Models: A Survey](https://mlsanity.com/paper/2409.19365/) (2024)
- [OmniControlNet: Dual-stage Integration for Conditional Image Generation](https://mlsanity.com/paper/2406.05871/) (2024)
- [Multi-turn Consistent Image Editing](https://mlsanity.com/paper/2505.04320/) (2025)
- [VINCIE: Unlocking In-context Image Editing from Video](https://mlsanity.com/paper/2506.10941/) (2025)
- [Style-A-Video: Agile Diffusion for Arbitrary Text-based Video Style Transfer](https://mlsanity.com/paper/2305.05464/) (2023)
- [Image Anything: Towards Reasoning-coherent and Training-free Multi-modal Image Generation](https://mlsanity.com/paper/2401.17664/) (2024)
- [Slight Corruption in Pre-training Data Makes Better Diffusion Models](https://mlsanity.com/paper/2405.20494/) (2024)
- [Vision + Language Applications: A Survey](https://mlsanity.com/paper/2305.14598/) (2023)
- [Isolated Diffusion: Optimizing Multi-Concept Text-to-Image Generation Training-Freely with Isolated Diffusion Guidance](https://mlsanity.com/paper/2403.16954/) (2024)
- [FreqEdit: Preserving High-Frequency Features for Robust Multi-Turn Image Editing](https://mlsanity.com/paper/2512.01755/) (2025)
- [DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing](https://mlsanity.com/paper/2506.02560/) (2025)
- [CompSlider: Compositional Slider for Disentangled Multiple-Attribute Image Generation](https://mlsanity.com/paper/2509.01028/) (2025)
- [CogniEdit: Dense Gradient Flow Optimization for Fine-Grained Image Editing](https://mlsanity.com/paper/2512.13276/) (2025)
- [Latent Inversion with Timestep-aware Sampling for Training-free Non-rigid Editing](https://mlsanity.com/paper/2402.08601/) (2024)
- [Unsupervised Region-Based Image Editing of Denoising Diffusion Models](https://mlsanity.com/paper/2412.12912/) (2024)
- [LLM-Enabled Style and Content Regularization for Personalized Text-to-Image Generation](https://mlsanity.com/paper/2504.15309/) (2025)
- [SeamCam: Quantifying Seamless Camouflage via Multi-Cue Visual Detectability](https://mlsanity.com/paper/2605.16515/) (2026)

## Abstract (arXiv)

Image manipulation under the guidance of textual descriptions has recently received a broad range of attention. In this study, we focus on the regional editing of images with the guidance of given text prompts. Different from current mask-based image editing methods, we propose a novel region-aware diffusion model (RDM) for entity-level image editing, which could automatically locate the region of interest and replace it following given text prompts. To strike a balance between image fidelity and inference speed, we design the intensive diffusion pipeline by combing latent space diffusion and enhanced directional guidance. In addition, to preserve image content in non-edited regions, we introduce regional-aware entity editing to modify the region of interest and preserve the out-of-interest region. We validate the proposed RDM beyond the baseline methods through extensive qualitative and quantitative experiments. The results show that RDM outperforms the previous approaches in terms of visual quality, overall harmonization, non-editing region content preservation, and text-image semantic consistency. The codes are available at https://github.com/haha-lisa/RDM-Region-Aware-Diffusion-Model.