# DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers arXiv: https://arxiv.org/abs/2505.21541 ยท HTML: https://mlsanity.com/paper/2505.21541/ ## Key facts - Cited by 20 papers in the map (data through 14 Oct 2026). - Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. - Builds on 25 of its 77 references that are in the map. - First posted on arXiv 24 May 2025. ## Placed in - Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/) ## Builds on (most cited first, 25 of 25) - [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023) - [MagicVideo: Efficient Video Generation With Latent Diffusion Models](https://mlsanity.com/paper/2211.11018/) (2022) - [OminiControl: Minimal and Universal Control for Diffusion Transformer](https://mlsanity.com/paper/2411.15098/) (2024) - [SegDiff: Image Segmentation with Diffusion Probabilistic Models](https://mlsanity.com/paper/2112.00390/) (2021) - [AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea](https://mlsanity.com/paper/2411.15738/) (2024) - [Inpaint Anything: Segment Anything Meets Image Inpainting](https://mlsanity.com/paper/2304.06790/) (2023) - [In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer](https://mlsanity.com/paper/2504.20690/) (2025) - [HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing](https://mlsanity.com/paper/2404.09990/) (2024) - [Transparent Image Layer Diffusion using Latent Transparency](https://mlsanity.com/paper/2402.17113/) (2024) - [MotionBooth: Motion-Aware Customized Text-to-Video Generation](https://mlsanity.com/paper/2406.17758/) (2024) - [Insert Anything: Image Insertion via In-Context Editing in DiT](https://mlsanity.com/paper/2504.15009/) (2025) - [LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation](https://mlsanity.com/paper/2408.13252/) (2024) - [MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation](https://mlsanity.com/paper/2502.01572/) (2025) - [PromptFix: You Prompt and We Fix the Photo](https://mlsanity.com/paper/2405.16785/) (2024) - [Generative Omnimatte: Learning to Decompose Video into Layers](https://mlsanity.com/paper/2411.16683/) (2024) - [Generative Image Layer Decomposition with Visual Effects](https://mlsanity.com/paper/2411.17864/) (2024) - [SmartEraser: Remove Anything from Images using Masked-Region Guidance](https://mlsanity.com/paper/2501.08279/) (2025) - [TransAnimate: Taming Layer Diffusion to Generate RGBA Video](https://mlsanity.com/paper/2503.17934/) (2025) - [Edit Transfer: Learning Image Editing via Vision In-Context Relations](https://mlsanity.com/paper/2503.13327/) (2025) - [Unified Dense Prediction of Video Diffusion](https://mlsanity.com/paper/2503.09344/) (2025) - [GarmentDiffusion: 3D Garment Sewing Pattern Generation with Multimodal Diffusion Transformers](https://mlsanity.com/paper/2504.21476/) (2025) - [PixelHacker: Image Inpainting with Structural and Semantic Consistency](https://mlsanity.com/paper/2504.20438/) (2025) - [Model See Model Do: Speech-Driven Facial Animation with Style Control](https://mlsanity.com/paper/2505.01319/) (2025) - [Foodfusion: A Novel Approach for Food Image Composition via Diffusion Models](https://mlsanity.com/paper/2408.14135/) (2024) - [Efficient Portrait Matte Creation With Layer Diffusion and Connectivity Priors](https://mlsanity.com/paper/2501.16147/) (2025) ## Built on by (most cited first, 20 of 20) - [Controllable Video Generation: A Survey](https://mlsanity.com/paper/2507.16869/) (2025) - [RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers](https://mlsanity.com/paper/2506.02528/) (2025) - [Mitty: Diffusion-based Human-to-Robot Video Generation](https://mlsanity.com/paper/2512.17253/) (2025) - [OmniPSD: Layered PSD Generation with Diffusion Transformer](https://mlsanity.com/paper/2512.09247/) (2025) - [WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generation](https://mlsanity.com/paper/2511.22098/) (2025) - [StreamingEffect: Real-Time Human-Centric Video Effect Generation](https://mlsanity.com/paper/2605.17019/) (2026) - [Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceivers](https://mlsanity.com/paper/2511.18673/) (2025) - [OmniRefiner: Reinforcement-Guided Local Diffusion Refinement](https://mlsanity.com/paper/2511.19990/) (2025) - [The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignment](https://mlsanity.com/paper/2511.20614/) (2025) - [NumeriKontrol: Adding Numeric Control to Diffusion Transformers for Instruction-based Image Editing](https://mlsanity.com/paper/2511.23105/) (2025) - [Loom: Diffusion-Transformer for Interleaved Generation](https://mlsanity.com/paper/2512.18254/) (2025) - [SIGMA: Selective-Interleaved Generation with Multi-Attribute Tokens](https://mlsanity.com/paper/2602.07564/) (2026) - [Unified Long Video Inpainting and Outpainting via Overlapping High-Order Co-Denoising](https://mlsanity.com/paper/2511.03272/) (2025) - [Image Generation as a Visual Planner for Robotic Manipulation](https://mlsanity.com/paper/2512.00532/) (2025) - [TokenPure: Watermark Removal through Tokenized Appearance and Structural Guidance](https://mlsanity.com/paper/2512.01314/) (2025) - [SWEET: Sparse World Modeling with Image Editing for Embodied Task Execution](https://mlsanity.com/paper/2605.19319/) (2026) - [ReDesign: Recovering Editable Design Structures from Images via Agentic Decomposition](https://mlsanity.com/paper/2607.25565/) (2026) - [Cycle-Consistent Tuning for Layered Image Decomposition](https://mlsanity.com/paper/2602.20989/) (2026) - [PAI-Actor: Cinematic Multi-Character Replacement in Dynamic Scenes](https://mlsanity.com/paper/2609.05918/) (2026) - [VectorHarness: Recovering Editable, Relation-Preserving Structure from Scientific Graphics](https://mlsanity.com/paper/2609.13294/) (2026) ## Abstract (arXiv) Diffusion models have recently motivated great success in many generation tasks like object removal. Nevertheless, existing image decomposition methods struggle to disentangle semi-transparent or transparent layer occlusions due to mask prior dependencies, static object assumptions, and the lack of datasets. In this paper, we delve into a novel task: Layer-Wise Decomposition of Alpha-Composited Images, aiming to recover constituent layers from single overlapped images under the condition of semi-transparent/transparent alpha layer non-linear occlusion. To address challenges in layer ambiguity, generalization, and data scarcity, we first introduce AlphaBlend, the first large-scale and high-quality dataset for transparent and semi-transparent layer decomposition, supporting six real-world subtasks (e.g., translucent flare removal, semi-transparent cell decomposition, glassware decomposition). Building on this dataset, we present DiffDecompose, a diffusion Transformer-based framework that learns the posterior over possible layer decompositions conditioned on the input image, semantic prompts, and blending type. Rather than regressing alpha mattes directly, DiffDecompose performs In-Context Decomposition, enabling the model to predict one or multiple layers without per-layer supervision, and introduces Layer Position Encoding Cloning to maintain pixel-level correspondence across layers. Extensive experiments on the proposed AlphaBlend dataset and public LOGO dataset verify the effectiveness of DiffDecompose. The code and dataset will be available upon paper acceptance. Our code will be available at: https://github.com/Wangzt1121/DiffDecompose.