# Object-Centric Image Generation from Layouts

arXiv: https://arxiv.org/abs/2003.07449 · HTML: https://mlsanity.com/paper/2003.07449/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Mar 2020.

## Built on by (most cited first, 30 of 57)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https://mlsanity.com/paper/2301.07093/) (2023)
- [Training-Free Layout Control with Cross-Attention Guidance](https://mlsanity.com/paper/2304.03373/) (2023)
- [BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion](https://mlsanity.com/paper/2307.10816/) (2023)
- [SpaText: Spatio-Textual Representation for Controllable Image Generation](https://mlsanity.com/paper/2211.14305/) (2022)
- [ReCo: Region-Controlled Text-to-Image Generation](https://mlsanity.com/paper/2211.15518/) (2022)
- [Instance-Conditioned GAN](https://mlsanity.com/paper/2109.05070/) (2021)
- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models](https://mlsanity.com/paper/2305.11281/) (2023)
- [Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis](https://mlsanity.com/paper/2208.13753/) (2022)
- [DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion model](https://mlsanity.com/paper/2310.07771/) (2023)
- [LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation](https://mlsanity.com/paper/2302.08908/) (2023)
- [Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training](https://mlsanity.com/paper/2211.11138/) (2022)
- [LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts](https://mlsanity.com/paper/2310.10640/) (2023)
- [Adversarial Text-to-Image Synthesis: A Review](https://mlsanity.com/paper/2101.09983/) (2021)
- [Modeling Image Composition for Complex Scene Generation](https://mlsanity.com/paper/2206.00923/) (2022)
- [SceneComposer: Any-Level Semantic Image Synthesis](https://mlsanity.com/paper/2211.11742/) (2022)
- [Context-Aware Layout to Image Generation with Enhanced Object Appearance](https://mlsanity.com/paper/2103.11897/) (2021)
- [SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis](https://mlsanity.com/paper/2304.14573/) (2023)
- [High-Resolution Complex Scene Synthesis with Transformers](https://mlsanity.com/paper/2105.06458/) (2021)
- [LoCo: Locally Constrained Training-Free Layout-to-Image Synthesis](https://mlsanity.com/paper/2311.12342/) (2023)
- [LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts](https://mlsanity.com/paper/2308.06713/) (2023)
- [Are These the Same Apple? Comparing Images Based on Object Intrinsics](https://mlsanity.com/paper/2311.00750/) (2023)
- [IFAdapter: Instance Feature Control for Grounded Text-to-Image Generation](https://mlsanity.com/paper/2409.08240/) (2024)
- [Image Scene Graph Generation (SGG) Benchmark](https://mlsanity.com/paper/2107.12604/) (2021)
- [CoGS: Controllable Generation and Search from Sketch and Style](https://mlsanity.com/paper/2203.09554/) (2022)
- [TISE: Bag of Metrics for Text-to-Image Synthesis Evaluation](https://mlsanity.com/paper/2112.01398/) (2021)
- [Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation](https://mlsanity.com/paper/2304.06671/) (2023)
- [Editable Image Elements for Controllable Synthesis](https://mlsanity.com/paper/2404.16029/) (2024)
- [Interactive Data Synthesis for Systematic Vision Adaptation via LLMs-AIGCs Collaboration](https://mlsanity.com/paper/2305.12799/) (2023)

## Abstract (arXiv)

Despite recent impressive results on single-object and single-domain image generation, the generation of complex scenes with multiple objects remains challenging. In this paper, we start with the idea that a model must be able to understand individual objects and relationships between objects in order to generate complex scenes well. Our layout-to-image-generation method, which we call Object-Centric Generative Adversarial Network (or OC-GAN), relies on a novel Scene-Graph Similarity Module (SGSM). The SGSM learns representations of the spatial relationships between objects in the scene, which lead to our model's improved layout-fidelity. We also propose changes to the conditioning mechanism of the generator that enhance its object instance-awareness. Apart from improving image quality, our contributions mitigate two failure modes in previous approaches: (1) spurious objects being generated without corresponding bounding boxes in the layout, and (2) overlapping bounding boxes in the layout leading to merged objects in images. Extensive quantitative evaluation and ablation studies demonstrate the impact of our contributions, with our model outperforming previous state-of-the-art approaches on both the COCO-Stuff and Visual Genome datasets. Finally, we address an important limitation of evaluation metrics used in previous works by introducing SceneFID -- an object-centric adaptation of the popular Fr'echet Inception Distance metric, that is better suited for multi-object images.