# Source Prompt Disentangled Inversion for Boosting Image Editability with Diffusion Models

arXiv: https://arxiv.org/abs/2403.11105 · HTML: https://mlsanity.com/paper/2403.11105/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 60 of its 62 references that are in the map.
- First posted on arXiv 17 Mar 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 60)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)

## Built on by (most cited first, 22 of 22)

- [RORem: Training a Robust Object Remover with Human-in-the-Loop](https://mlsanity.com/paper/2501.00740/) (2025)
- [ChordEdit: One-Step Low-Energy Transport for Image Editing](https://mlsanity.com/paper/2602.19083/) (2026)
- [Fast Multi-view Consistent 3D Editing with Video Priors](https://mlsanity.com/paper/2511.23172/) (2025)
- [PostEdit: Posterior Sampling for Efficient Zero-Shot Image Editing](https://mlsanity.com/paper/2410.04844/) (2024)
- [Omni-3DEdit: Generalized Versatile 3D Editing in One-Pass](https://mlsanity.com/paper/2603.17841/) (2026)
- [Early Timestep Zero-Shot Candidate Selection for Instruction-Guided Image Editing](https://mlsanity.com/paper/2504.13490/) (2025)
- [Cora: Correspondence-aware image editing using few step diffusion](https://mlsanity.com/paper/2505.23907/) (2025)
- [DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing](https://mlsanity.com/paper/2506.02560/) (2025)
- [Oscillation Inversion: Understand the structure of Large Flow Model through the Lens of Inversion Method](https://mlsanity.com/paper/2411.11135/) (2024)
- [SyncNoise: Geometrically Consistent Noise Prediction for Text-based 3D Scene Editing](https://mlsanity.com/paper/2406.17396/) (2024)
- [Task-Oriented Diffusion Inversion for High-Fidelity Text-based Editing](https://mlsanity.com/paper/2408.13395/) (2024)
- [Free Lunch for Stabilizing Rectified Flow Inversion](https://mlsanity.com/paper/2602.11850/) (2026)
- [WhereEdit: Mask-aware Local Latent Editing for One-Step Image Editing](https://mlsanity.com/paper/2607.20883/) (2026)
- [Tuning-Free Image Editing with Fidelity and Editability via Unified Latent Diffusion Model](https://mlsanity.com/paper/2504.05594/) (2025)
- [Do Concept Replacement Techniques Really Erase Unacceptable Concepts?](https://mlsanity.com/paper/2506.08991/) (2025)
- [Improving Diffusion-Based Image Editing Faithfulness via Guidance and Scheduling](https://mlsanity.com/paper/2506.21045/) (2025)
- [FBSDiff++: Improved Frequency Band Substitution of Diffusion Features for Efficient and Highly Controllable Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2601.19115/) (2026)
- [Dual Inversion for Text-to-Image Diffusion Models: From Both Prompt and Noise Perspectives](https://mlsanity.com/paper/2607.26735/) (2026)
- [MILD: Multi-Layer Diffusion Strategy for Complex and Precise Multi-IP Aware Human Erasing](https://mlsanity.com/paper/2508.06543/) (2025)
- [APPLE: Attribute-Preserving Pseudo-Labeling for Diffusion-Based Face Swapping](https://mlsanity.com/paper/2601.15288/) (2026)
- [ResEdit: Residual embeddings for precise generative image editing](https://mlsanity.com/paper/2606.16457/) (2026)
- [Co-policy: Responsive Human-Robot Co-Creation for Musical Performances](https://mlsanity.com/paper/2606.19914/) (2026)

## Abstract (arXiv)

Text-driven diffusion models have significantly advanced the image editing performance by using text prompts as inputs. One crucial step in text-driven image editing is to invert the original image into a latent noise code conditioned on the source prompt. While previous methods have achieved promising results by refactoring the image synthesizing process, the inverted latent noise code is tightly coupled with the source prompt, limiting the image editability by target text prompts. To address this issue, we propose a novel method called Source Prompt Disentangled Inversion (SPDInv), which aims at reducing the impact of source prompt, thereby enhancing the text-driven image editing performance by employing diffusion models. To make the inverted noise code be independent of the given source prompt as much as possible, we indicate that the iterative inversion process should satisfy a fixed-point constraint. Consequently, we transform the inversion problem into a searching problem to find the fixed-point solution, and utilize the pre-trained diffusion models to facilitate the searching process. The experimental results show that our proposed SPDInv method can effectively mitigate the conflicts between the target editing prompt and the source prompt, leading to a significant decrease in editing artifacts. In addition to text-driven image editing, with SPDInv we can easily adapt customized image generation models to localized editing tasks and produce promising performance. The source code are available at https://github.com/leeruibin/SPDInv.