# Generating Multiple Objects at Spatially Distinct Locations

arXiv: https://arxiv.org/abs/1901.00686 · HTML: https://mlsanity.com/paper/1901.00686/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2019.

## Built on by (most cited first, 30 of 41)

- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Training-Free Layout Control with Cross-Attention Guidance](https://mlsanity.com/paper/2304.03373/) (2023)
- [Cross-Modal Contrastive Learning for Text-to-Image Generation](https://mlsanity.com/paper/2101.04702/) (2021)
- [MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space](https://mlsanity.com/paper/2105.01879/) (2021)
- [SpaText: Spatio-Textual Representation for Controllable Image Generation](https://mlsanity.com/paper/2211.14305/) (2022)
- [ReCo: Region-Controlled Text-to-Image Generation](https://mlsanity.com/paper/2211.15518/) (2022)
- [Illiterate DALL-E Learns to Compose](https://mlsanity.com/paper/2110.11405/) (2021)
- [Can multi-label classification networks know what they don't know?](https://mlsanity.com/paper/2109.14162/) (2021)
- [LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation](https://mlsanity.com/paper/2308.05095/) (2023)
- [Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wild](https://mlsanity.com/paper/2203.03800/) (2022)
- [Freestyle Layout-to-Image Synthesis](https://mlsanity.com/paper/2303.14412/) (2023)
- [Improving Text-to-Image Synthesis Using Contrastive Learning](https://mlsanity.com/paper/2107.02423/) (2021)
- [Adversarial Text-to-Image Synthesis: A Review](https://mlsanity.com/paper/2101.09983/) (2021)
- [SceneComposer: Any-Level Semantic Image Synthesis](https://mlsanity.com/paper/2211.11742/) (2022)
- [Multi-modal Machine Learning in Engineering Design: A Review and Future Directions](https://mlsanity.com/paper/2302.10909/) (2023)
- [Collaging Class-specific GANs for Semantic Image Synthesis](https://mlsanity.com/paper/2110.04281/) (2021)
- [Compositional Transformers for Scene Generation](https://mlsanity.com/paper/2111.08960/) (2021)
- [DSE-GAN: Dynamic Semantic Evolution Generative Adversarial Network for Text-to-Image Generation](https://mlsanity.com/paper/2209.01339/) (2022)
- [Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation](https://mlsanity.com/paper/2304.06671/) (2023)
- [AttrLostGAN: Attribute Controlled Image Synthesis from Reconfigurable Layout and Style](https://mlsanity.com/paper/2103.13722/) (2021)
- [SIEDOB: Semantic Image Editing by Disentangling Object and Background](https://mlsanity.com/paper/2303.13062/) (2023)
- [DR-GAN: Distribution Regularization for Text-to-Image Generation](https://mlsanity.com/paper/2204.07945/) (2022)
- [CharacterGAN: Few-Shot Keypoint Character Animation and Reposing](https://mlsanity.com/paper/2102.03141/) (2021)
- [Exploiting Relationship for Complex-scene Image Generation](https://mlsanity.com/paper/2104.00356/) (2021)
- [Deep Image Synthesis from Intuitive User Input: A Review and Perspectives](https://mlsanity.com/paper/2107.04240/) (2021)
- [Markup-to-Image Diffusion Models with Scheduled Sampling](https://mlsanity.com/paper/2210.05147/) (2022)
- [T2IW: Joint Text to Image & Watermark Generation](https://mlsanity.com/paper/2309.03815/) (2023)
- [DT2I: Dense Text-to-Image Generation from Region Descriptions](https://mlsanity.com/paper/2204.02035/) (2022)
- [Text-to-Image Generation via Implicit Visual Guidance and Hypernetwork](https://mlsanity.com/paper/2208.08493/) (2022)
- [ALR-GAN: Adaptive Layout Refinement for Text-to-Image Synthesis](https://mlsanity.com/paper/2304.06297/) (2023)

## Abstract (arXiv)

Recent improvements to Generative Adversarial Networks (GANs) have made it possible to generate realistic images in high resolution based on natural language descriptions such as image captions. Furthermore, conditional GANs allow us to control the image generation process through labels or even natural language descriptions. However, fine-grained control of the image layout, i.e. where in the image specific objects should be located, is still difficult to achieve. This is especially true for images that should contain multiple distinct objects at different spatial locations. We introduce a new approach which allows us to control the location of arbitrarily many objects within an image by adding an object pathway to both the generator and the discriminator. Our approach does not need a detailed semantic layout but only bounding boxes and the respective labels of the desired objects are needed. The object pathway focuses solely on the individual objects and is iteratively applied at the locations specified by the bounding boxes. The global pathway focuses on the image background and the general image layout. We perform experiments on the Multi-MNIST, CLEVR, and the more complex MS-COCO data set. Our experiments show that through the use of the object pathway we can control object locations within images and can model complex scenes with multiple objects at various locations. We further show that the object pathway focuses on the individual objects and learns features relevant for these, while the global pathway focuses on global image characteristics and the image background.