# MDP: A Generalized Framework for Text-Guided Image Editing by Manipulating the Diffusion Path

arXiv: https://arxiv.org/abs/2303.16765 · HTML: https://mlsanity.com/paper/2303.16765/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 59 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 30 of 51)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)

## Built on by (most cited first, 18 of 18)

- [Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editing](https://mlsanity.com/paper/2309.15664/) (2023)
- [InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructions](https://mlsanity.com/paper/2305.18047/) (2023)
- [UniEdit-Flow: Unleashing Inversion and Editing in the Era of Flow Models](https://mlsanity.com/paper/2504.13109/) (2025)
- [LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning](https://mlsanity.com/paper/2312.03849/) (2023)
- [LIME: Localized Image Editing via Attention Regularization in Diffusion Models](https://mlsanity.com/paper/2312.09256/) (2023)
- [IMPUS: Image Morphing with Perceptually-Uniform Sampling Using Diffusion Models](https://mlsanity.com/paper/2311.06792/) (2023)
- [FRDiff : Feature Reuse for Universal Training-free Acceleration of Diffusion Models](https://mlsanity.com/paper/2312.03517/) (2023)
- [LocInv: Localization-aware Inversion for Text-Guided Image Editing](https://mlsanity.com/paper/2405.01496/) (2024)
- [IterInv: Iterative Inversion for Pixel-Level T2I Models](https://mlsanity.com/paper/2310.19540/) (2023)
- [MedEBench: Diagnosing Reliability in Text-Guided Medical Image Editing](https://mlsanity.com/paper/2506.01921/) (2025)
- [ParSEL: Parameterized Shape Editing with Language](https://mlsanity.com/paper/2405.20319/) (2024)
- [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024)
- [EditCLIP: Representation Learning for Image Editing](https://mlsanity.com/paper/2503.20318/) (2025)
- [Diffusion Model Compression for Image-to-Image Translation](https://mlsanity.com/paper/2401.17547/) (2024)
- [LOCATEdit: Graph Laplacian Optimized Cross Attention for Localized Text-Guided Image Editing](https://mlsanity.com/paper/2503.21541/) (2025)
- [UIP2P: Unsupervised Instruction-based Image Editing via Edit Reversibility Constraint](https://mlsanity.com/paper/2412.15216/) (2024)
- [PromptArtisan: Multi-instruction Image Editing in Single Pass with Complete Attention Control](https://mlsanity.com/paper/2502.10258/) (2025)
- [Towards Generalized and Training-Free Text-Guided Semantic Manipulation](https://mlsanity.com/paper/2504.17269/) (2025)

## Abstract (arXiv)

Image generation using diffusion can be controlled in multiple ways. In this paper, we systematically analyze the equations of modern generative diffusion networks to propose a framework, called MDP, that explains the design space of suitable manipulations. We identify 5 different manipulations, including intermediate latent, conditional embedding, cross attention maps, guidance, and predicted noise. We analyze the corresponding parameters of these manipulations and the manipulation schedule. We show that some previous editing methods fit nicely into our framework. Particularly, we identified one specific configuration as a new type of control by manipulating the predicted noise, which can perform higher-quality edits than previous work for a variety of local and global edits.