mlsanityData to 7 Oct 2026

Text2Layer: Layered Image Generation using Latent Diffusion Model

arXiv 2307.09781 ↗First posted 19 Jul 2023cs.CV cs.AI cs.LGcited by 35

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Layer compositing is one of the most popular image editing workflows among both amateurs and professionals. Motivated by the success of diffusion models, we explore layer compositing from a layered image generation perspective. Instead of generating an image, we propose to generate background, foreground, layer mask, and the composed image simultaneously. To achieve layered image generation, we train an autoencoder that is able to reconstruct layered images and train diffusion models on the latent representation. One benefit of the proposed problem is to enable better compositing workflows in addition to the high-quality image output. Another benefit is producing higher-quality layer masks compared to masks produced by a separate step of image segmentation. Experimental results show that the proposed method is able to generate high-quality layered images and initiates a benchmark for future work.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 56 of its 66 references that are in the map. First posted on arXiv 19 Jul 2023.

Builds on

56 of 66 references.

Closest ancestors · 10

Same direction, newest first

  1. 2022SINE: SINgle Image Editing with Text-to-Image Diffusion Models8 Dec
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  3. 2022Direct Inversion: Optimization-Free Text-Driven Real Image Editing with Diffusion Models15 Nov
  4. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  9. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov
  10. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation6 Oct

Other references · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  9. 2020Denoising Diffusion Implicit Modelscited by 9,655
  10. 2017Mask R-CNNcited by 7,259
  11. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
34 more
  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  5. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  7. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  9. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  10. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  11. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  12. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  13. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  14. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  15. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  16. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  17. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  18. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  19. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  20. 2016Deeply supervised salient object detection with short connectionscited by 196
  21. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  22. 2015Visual Saliency Based on Multiscale Deep Featurescited by 185
  23. 2019Salient Object Detection in the Deep Learning Era: An In-Depth Surveycited by 163
  24. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  25. 2017Deep Image Mattingcited by 150
  26. 2021Salient Object Detection via Integrity Learningcited by 133
  27. 2020MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decompositioncited by 127
  28. 2020Global Context-Aware Progressive Aggregation Network for Salient Object Detectioncited by 109
  29. 2020Bridging Composite and Real: Towards End-to-end Deep Image Mattingcited by 89
  30. 2020Mask Guided Matting via Progressive Refinement Networkcited by 84
  31. 2019Context-Aware Image Matting for Simultaneous Foreground and Alpha Estimationcited by 79
  32. 2020Natural Image Matting via Guided Contextual Attentioncited by 78
  33. 2021Semantic Image Mattingcited by 55
  34. 2022MatteFormer: Transformer-Based Image Matting via Prior-Tokenscited by 48

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 17; 2026: 10.

35 papers.

Sort:

By area:

  1. 2024Transparent Image Layer Diffusion using Latent Transparencycited by 111
  2. 2024MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generationcited by 38
  3. 2025ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generationcited by 37
  4. 2025Qwen-Image-Layered: Towards Inherent Editability via Layer Decompositioncited by 29
  5. 2024LayerDiff: Exploring Text-guided Multi-layered Composable Image Synthesis via Layer-Collaborative Diffusion Modelcited by 26
  6. 2025LayerD: Decomposing Raster Graphic Designs into Layerscited by 20
  7. 2025DreamLayer: Simultaneous Multi-Layer Generation via Diffusion Modecited by 17
  8. 2025PrismLayers: Open Data for High-Quality Multi-Layer Transparent Image Generative Modelscited by 14
  9. 2024Generating Compositional Scenes via Text-to-image RGBA Instance Generationcited by 13
  10. 2024TKG-DM: Training-free Chroma Key Content Generation Diffusion Modelcited by 11
  11. 2025PSDiffusion: Harmonized Multi-Layer Image Generation via Layout and Appearance Alignmentcited by 11
  12. 2025OmniPSD: Layered PSD Generation with Diffusion Transformercited by 11
23 more
  1. 2025Rethinking Layered Graphic Design Generation with a Top-Down Approachcited by 9
  2. 2024DreamDissector: Learning Disentangled Text-to-3D Generation from 2D Diffusion Priorscited by 8
  3. 2025AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learningcited by 7
  4. 2025Improving Editability in Image Generation with Layer-wise Memorycited by 5
  5. 2025Trans-Adapter: A Plug-and-Play Framework for Transparent Image Inpaintingcited by 5
  6. 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modelingcited by 5
  7. 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cuescited by 4
  8. 2025Multi-Agent Amodal Completion: Direct Synthesis with Fine-Grained Semantic Guidancecited by 3
  9. 2026MiLDEdit: Reasoning-Based Multi-Layer Design Document Editingcited by 3
  10. 2026Referring Layer Decompositioncited by 3
  11. 2024Data Augmentation in Human-Centric Visioncited by 2
  12. 2025LayerT2V: A Unified Multi-Layer Video Generation Frameworkcited by 2
  13. 2025TAUE: Training-free Noise Transplant and Cultivation Diffusion Modelcited by 2
  14. 2024GLoD: Composing Global Contexts and Local Details in Image Generationcited by 1
  15. 2025LumiGen: An LVLM-Enhanced Iterative Framework for Fine-Grained Text-to-Image Generationcited by 1
  16. 2026Does Synthetic Layered Design Data Benefit Layered Design Decomposition?cited by 1
  17. 2026ReDesign: Recovering Editable Design Structures from Images via Agentic Decompositioncited by 1
  18. 2025RaDL: Relation-aware Disentangled Learning for Multi-Instance Text-to-Image Generationcited by 0
  19. 2026A Unified and Controllable Framework for Layered Image Generation with Visual Effectscited by 0
  20. 2026TransText: Alpha-as-RGB Representation for Transparent Text Animationcited by 0
  21. 2026LimeCross: Context-Conditioned Layered Image Editing with Structural Consistencycited by 0
  22. 2026LiWi: Layering in the Wildcited by 0
  23. 2026PixelART: Image-to-Layer Decomposition without Latents or Text-to-Image Pretrainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LayerFusion: Harmonized Multi-Layer Text-to-Image Generation with Generative Priors
  2. 2023LayerDiffusion: Layered Controlled Image Editing with Diffusion Models
  3. 2024Streamlining Image Editing with Layered Diffusion Brushes
  4. 2023Text-driven Visual Synthesis with Latent Diffusion Prior
  5. 2026MRT: Masked Region Transformer for Layered Image Generation and Editing at Scale
  6. 2023Collage Diffusion
  7. 2025LayerSync: Self-aligning Intermediate Layers
  8. 2024Move Anything with Layered Scene Diffusion