Learning to Generate Multiple Objects from Dense and Occluded Layouts
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Text-to-image diffusion models fail to generate correct object counts in dense scenes, where overlapping instances collapse into indistinguishable structures despite appearing visually plausible. We identify this as instance ownership collapse: tokens from overlapping objects interact freely through attention, while heavily occluded instances receive weak supervision due to their small visible areas. We address this through layout-aware attention biases that softly bias token interactions toward region-consistent grouping and suppress cross-instance leakage, paired with an amodal-balanced loss that amplifies gradients for occluded objects based on their occlusion level. To enable systematic evaluation, we introduce OverlapDepth-45K, a benchmark of densely overlapping scenes with amodal supervision. Our approach substantially improves count accuracy and prevents instance merging while preserving image quality. Project page: https://bachngoh.github.io/AIBL
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 22 of its 26 references that are in the map. First posted on arXiv 3 Jul 2026.
Builds on
22 of 26 references.
Closest ancestors · 11
Same direction, newest first
- 2024Make It Count: Text-to-Image Generation with an Accurate Number of Objects14 Jun
- 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models16 Mar
- 2024InstanceDiffusion: Instance-level Control for Image Generation5 Feb
- 2023Reason out Your Layout: Evoking the Layout Master from Large Language Models for Text-to-Image Synthesis28 Nov
- 2023Counting Guidance for High Fidelity Text-to-Image Synthesis30 Jun
- 2023Grounded Text-to-Image Synthesis with Attention Refocusing8 Jun
- 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation30 Mar
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
- 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
Other references · 11
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2025SAM 3: Segment Anything with Conceptscited by 835
- 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
- 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 247
- 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
- 2025OverLayBench: A Benchmark for Layout-to-Image Generation with Dense Overlapscited by 9
- 2026SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generationcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026InstancePin: Instance-Addressable Layout-to-Image Diffusion via Coordinate Pinning
- 2026OccluRank: Controllable Occlusion-Aware Layout-to-Image Generation by Adding Just an Ordinal Rank
- 2025CountLoop: Training-Free High-Instance Image Generation via Iterative Agent Guidance
- 2025TRACE: Your Diffusion Model is Secretly an Instance Edge Detector
- 2023LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts
- 2024ObjectMate: A Recurrence Prior for Object Insertion and Subject-Driven Generation
- 2026OcclusionFormer: Arranging Z-Order for Layout-Grounded Image Generation
- 2026Layer-wise Instance Binding for Regional and Occlusion Control in Text-to-Image Diffusion Transformers