# ALR-GAN: Adaptive Layout Refinement for Text-to-Image Synthesis

arXiv: https://arxiv.org/abs/2304.06297 · HTML: https://mlsanity.com/paper/2304.06297/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 52 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 30 of 33)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis](https://mlsanity.com/paper/1904.01310/) (2019)
- [Cross-Modal Contrastive Learning for Text-to-Image Generation](https://mlsanity.com/paper/2101.04702/) (2021)
- [StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1710.10916/) (2017)
- [Controllable Text-to-Image Generation](https://mlsanity.com/paper/1909.07083/) (2019)
- [MirrorGAN: Learning Text-to-image Generation by Redescription](https://mlsanity.com/paper/1903.05854/) (2019)
- [Dual-Path Convolutional Image-Text Embeddings with Instance Loss](https://mlsanity.com/paper/1711.05535/) (2017)
- [StoryGAN: A Sequential Conditional GAN for Story Visualization](https://mlsanity.com/paper/1812.02784/) (2018)
- [Object-driven Text-to-Image Synthesis via Adversarial Training](https://mlsanity.com/paper/1902.10740/) (2019)
- [Image Generation from Layout](https://mlsanity.com/paper/1811.11389/) (2018)
- [Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis](https://mlsanity.com/paper/1801.05091/) (2018)
- [Semantic Object Accuracy for Generative Text-to-Image Synthesis](https://mlsanity.com/paper/1910.13321/) (2019)
- [Image Synthesis From Reconfigurable Layout and Style](https://mlsanity.com/paper/1908.07500/) (2019)
- [Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Network](https://mlsanity.com/paper/1802.09178/) (2018)
- [Semantics Disentangling for Text-to-Image Generation](https://mlsanity.com/paper/1904.01480/) (2019)
- [Object-Centric Image Generation from Layouts](https://mlsanity.com/paper/2003.07449/) (2020)
- [Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models](https://mlsanity.com/paper/1711.06420/) (2017)
- [Learning Layout and Style Reconfigurable GANs for Controllable Image Synthesis](https://mlsanity.com/paper/2003.11571/) (2020)
- [DAE-GAN: Dynamic Aspect-aware GAN for Text-to-Image Synthesis](https://mlsanity.com/paper/2108.12141/) (2021)
- [PasteGAN: A Semi-Parametric Method to Generate Image from Scene Graph](https://mlsanity.com/paper/1905.01608/) (2019)
- [Generating Multiple Objects at Spatially Distinct Locations](https://mlsanity.com/paper/1901.00686/) (2019)
- [Describe What to Change: A Text-guided Unsupervised Image-to-Image Translation Approach](https://mlsanity.com/paper/2008.04200/) (2020)
- [VSE++: Improving Visual-Semantic Embeddings with Hard Negatives](https://mlsanity.com/paper/1707.05612/) (2017)
- [ChatPainter: Improving Text to Image Generation using Dialogue](https://mlsanity.com/paper/1802.08216/) (2018)

## Built on by (most cited first, 4 of 4)

- [coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generation](https://mlsanity.com/paper/2603.12829/) (2026)
- [Self-supervised Photographic Image Layout Representation Learning](https://mlsanity.com/paper/2403.03740/) (2024)
- [TraDiffusion: Trajectory-Based Training-Free Image Generation](https://mlsanity.com/paper/2408.09739/) (2024)
- [End-to-end Training for Text-to-Image Synthesis using Dual-Text Embeddings](https://mlsanity.com/paper/2502.01507/) (2025)

## Abstract (arXiv)

We propose a novel Text-to-Image Generation Network, Adaptive Layout Refinement Generative Adversarial Network (ALR-GAN), to adaptively refine the layout of synthesized images without any auxiliary information. The ALR-GAN includes an Adaptive Layout Refinement (ALR) module and a Layout Visual Refinement (LVR) loss. The ALR module aligns the layout structure (which refers to locations of objects and background) of a synthesized image with that of its corresponding real image. In ALR module, we proposed an Adaptive Layout Refinement (ALR) loss to balance the matching of hard and easy features, for more efficient layout structure matching. Based on the refined layout structure, the LVR loss further refines the visual representation within the layout area. Experimental results on two widely-used datasets show that ALR-GAN performs competitively at the Text-to-Image generation task.