mlsanityData to 14 Oct 2026

LayerDiffusion: Layered Controlled Image Editing with Diffusion Models

arXiv 2305.18676 ↗First posted 30 May 2023cs.CVcited by 40

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Text-guided image editing has recently experienced rapid development. However, simultaneously performing multiple editing actions on a single image, such as background replacement and specific subject attribute changes, while maintaining consistency between the subject and the background remains challenging. In this paper, we propose LayerDiffusion, a semantic-based layered controlled image editing method. Our method enables non-rigid editing and attribute modification of specific subjects while preserving their unique characteristics and seamlessly integrating them into new backgrounds. We leverage a large-scale text-to-image model and employ a layered controlled optimization strategy combined with layered diffusion training. During the diffusion process, an iterative guidance strategy is used to generate a final image that aligns with the textual description. Experimental results demonstrate the effectiveness of our method in generating highly coherent images that closely align with the given textual description. The edited images maintain a high similarity to the features of the input image and surpass the performance of current leading image editing methods. LayerDiffusion opens up new possibilities for controllable image editing.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 37 of its 40 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

37 of 40 references.

Closest ancestors · 13

Same direction, newest first

  1. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation16 Feb
  2. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  3. 2022SINE: SINgle Image Editing with Text-to-Image Diffusion Models8 Dec
  4. 2022SpaText: Spatio-Textual Representation for Controllable Image Generation25 Nov
  5. 2022DiffEdit: Diffusion-based semantic image editing with mask guidance20 Oct
  6. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  10. 2022Dual Diffusion Implicit Bridges for Image-to-Image Translation16 Mar
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  12. 2021Palette: Image-to-Image Diffusion Models10 Nov
1 more
  1. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation6 Oct

Other references · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2023Segment Anythingcited by 8,234
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  12. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
12 more
  1. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  2. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  3. 2022Denoising Diffusion Restoration Modelscited by 963
  4. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  5. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 695
  6. 2021Score-based Generative Modeling in Latent Spacecited by 636
  7. 2020GANSpace: Discovering Interpretable GAN Controlscited by 614
  8. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 349
  9. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  10. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  11. 2021Diffusion Models for Implicit Image Segmentation Ensemblescited by 198
  12. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 21; 2025: 12; 2026: 5.

40 papers.

Sort:

By area:

  1. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  2. 2024Transparent Image Layer Diffusion using Latent Transparencycited by 114
  3. 2023Improving Tuning-Free Real Image Editing with Proximal Guidancecited by 87
  4. 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformercited by 56
  5. 2024Stable-Makeup: When Real-World Makeup Transfer Meets Diffusion Modelcited by 45
  6. 2024MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generationcited by 38
  7. 2024Stable-Hair: Real-World Hair Transfer via Diffusion Modelcited by 35
  8. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  9. 2024Tuning-Free Image Customization with Image and Text Guidancecited by 25
  10. 2023IMPUS: Image Morphing with Perceptually-Uniform Sampling Using Diffusion Modelscited by 21
  11. 2025ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motionscited by 19
  12. 2024Pruning then Reweighting: Towards Data-Efficient Training of Diffusion Modelscited by 17
28 more
  1. 2024Instruction-based Image Manipulation by Watching How Things Movecited by 12
  2. 2025ForgetMe: Evaluating Selective Forgetting in Generative Modelscited by 11
  3. 2024ART3D: 3D Gaussian Splatting for Text-Guided Artistic Scenes Generationcited by 9
  4. 2024TALE: Training-free Cross-domain Image Composition via Adaptive Latent Manipulation and Energy-guided Optimizationcited by 9
  5. 2025LayerEdit: Disentangled Multi-Object Editing via Conflict-Aware Multi-Layer Learningcited by 7
  6. 2024Sketch-to-Architecture: Generative AI-aided Architectural Designcited by 4
  7. 2024ColorEdit: Training-free Image-Guided Color editing with diffusion modelcited by 4
  8. 2025MDE-Edit: Masked Dual-Editing for Multi-Object Image Editing via Diffusion Modelscited by 4
  9. 2025AIComposer: Any Style and Content Image Composition via Feature Integrationcited by 4
  10. 2025LDGen: Enhancing Text-to-Image Synthesis via Large Language Model-Driven Language Representationcited by 3
  11. 2025Zero-Shot Subject-Centric Generation for Creative Application Using Entropy Fusioncited by 3
  12. 2025CanvasComposer: Personalized Group Photo Generation via a Multi-Reference Canvascited by 3
  13. 2024Generating Daylight-driven Architectural Design via Diffusion Modelscited by 2
  14. 2025InstructAttribute: Fine-grained Object Attributes editing with Instructioncited by 2
  15. 2025Stable-Hair v2: Real-World Hair Transfer via Multiple-View Diffusion Modelcited by 2
  16. 2024Latent Inversion with Timestep-aware Sampling for Training-free Non-rigid Editingcited by 1
  17. 2024GLoD: Composing Global Contexts and Local Details in Image Generationcited by 1
  18. 2024Challenges and Opportunities in 3D Content Generationcited by 1
  19. 2024SIn-NeRF2NeRF: Editing 3D Scenes with Instructions through Segmentation and Inpaintingcited by 1
  20. 2024Self-Supervised Vision Transformer for Enhanced Virtual Clothes Try-Oncited by 0
  21. 2024Pinco: Position-induced Consistent Adapter for Diffusion Transformer in Foreground-conditioned Inpaintingcited by 0
  22. 2025Controllable Coupled Image Generation via Diffusion Modelscited by 0
  23. 2025Text-Conditioned Background Generation for Editable Multi-Layer Documentscited by 0
  24. 2026Mitigating Instance Entanglement in Instance-Dependent Partial Label Learningcited by 0
  25. 2026MakeupMirror: Improving Facial Attribute Preservation in Diffusion Models for Makeup Transfercited by 0
  26. 2026MagicMakeup: A Region-Controllable Diffusion Transformer for High-Fidelity Makeup-Transfercited by 0
  27. 2026X-MULTI: VLM-based Imaging Factor Disentanglement for Factor-Aware Image Synthesiscited by 0
  28. 2026DecFlowEdit: Self-Localized Flow-based Image Editing via Guidance Decouplingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LayerDiff: Exploring Text-guided Multi-layered Composable Image Synthesis via Layer-Collaborative Diffusion Model
  2. 2023Text2Layer: Layered Image Generation using Latent Diffusion Model
  3. 2023Continuous Layout Editing of Single Images with Diffusion Models
  4. 2024Streamlining Image Editing with Layered Diffusion Brushes
  5. 2024Move Anything with Layered Scene Diffusion
  6. 2023Layered Rendering Diffusion Model for Controllable Zero-Shot Image Synthesis
  7. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidance
  8. 2023PRedItOR: Text Guided Image Editing with Diffusion Prior