# DAE-GAN: Dynamic Aspect-aware GAN for Text-to-Image Synthesis

arXiv: https://arxiv.org/abs/2108.12141 · HTML: https://mlsanity.com/paper/2108.12141/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 45 references that are in the map.
- First posted on arXiv 27 Aug 2021.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis](https://mlsanity.com/paper/1904.01310/) (2019)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1710.10916/) (2017)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [MirrorGAN: Learning Text-to-image Generation by Redescription](https://mlsanity.com/paper/1903.05854/) (2019)
- [Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings](https://mlsanity.com/paper/1803.08495/) (2018)
- [Generating Images from Captions with Attention](https://mlsanity.com/paper/1511.02793/) (2015)
- [Knowing What, How and Why: A Near Complete Solution for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1911.01616/) (2019)
- [Object-driven Text-to-Image Synthesis via Adversarial Training](https://mlsanity.com/paper/1902.10740/) (2019)
- [Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis](https://mlsanity.com/paper/1801.05091/) (2018)
- [Image Synthesis From Reconfigurable Layout and Style](https://mlsanity.com/paper/1908.07500/) (2019)
- [Parallel Multiscale Autoregressive Density Estimation](https://mlsanity.com/paper/1703.03664/) (2017)
- [Text Guided Person Image Synthesis](https://mlsanity.com/paper/1904.05118/) (2019)
- [How to make a pizza: Learning a compositional layer-based GAN model](https://mlsanity.com/paper/1906.02839/) (2019)
- [TediGAN: Text-Guided Diverse Face Image Generation and Manipulation](https://mlsanity.com/paper/2012.03308/) (2020)
- [Realistic Image Generation using Region-phrase Attention](https://mlsanity.com/paper/1902.05395/) (2019)

## Built on by (most cited first, 30 of 45)

- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.14704/) (2022)
- [Vision-Language Models in Remote Sensing: Current Progress and Future Trends](https://mlsanity.com/paper/2305.05726/) (2023)
- [GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesis](https://mlsanity.com/paper/2301.12959/) (2023)
- [Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2304.01816/) (2023)
- [Improved Vector Quantized Diffusion Models](https://mlsanity.com/paper/2205.16007/) (2022)
- [CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Model](https://mlsanity.com/paper/2403.11614/) (2024)
- [Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Model](https://mlsanity.com/paper/2501.00895/) (2025)
- [Instant3D: Instant Text-to-3D Generation](https://mlsanity.com/paper/2311.08403/) (2023)
- [Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation](https://mlsanity.com/paper/2206.07771/) (2022)
- [X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance](https://mlsanity.com/paper/2303.15764/) (2023)
- [StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis](https://mlsanity.com/paper/2203.15799/) (2022)
- [Txt2Img-MHN: Remote Sensing Image Generation from Text Using Modern Hopfield Networks](https://mlsanity.com/paper/2208.04441/) (2022)
- [Draw Your Art Dream: Diverse Digital Art Synthesis with Multimodal Guided Diffusion](https://mlsanity.com/paper/2209.13360/) (2022)
- [RSDiff: Remote Sensing Image Generation from Text Using Diffusion Model](https://mlsanity.com/paper/2309.02455/) (2023)
- [DreamIdentity: Improved Editability for Efficient Face-identity Preserved Image Generation](https://mlsanity.com/paper/2307.00300/) (2023)
- [Attribute-Centric Compositional Text-to-Image Generation](https://mlsanity.com/paper/2301.01413/) (2023)
- [Fine-grained Audible Video Description](https://mlsanity.com/paper/2303.15616/) (2023)
- [DSE-GAN: Dynamic Semantic Evolution Generative Adversarial Network for Text-to-Image Generation](https://mlsanity.com/paper/2209.01339/) (2022)
- [Recurrent Affine Transformation for Text-to-image Synthesis](https://mlsanity.com/paper/2204.10482/) (2022)
- [On the Cultural Gap in Text-to-Image Generation](https://mlsanity.com/paper/2307.02971/) (2023)
- [Layout-Bridging Text-to-Image Synthesis](https://mlsanity.com/paper/2208.06162/) (2022)
- [Conditioning Diffusion Models via Attributes and Semantic Masks for Face Generation](https://mlsanity.com/paper/2306.00914/) (2023)
- [CogCartoon: Towards Practical Story Visualization](https://mlsanity.com/paper/2312.10718/) (2023)
- [Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation](https://mlsanity.com/paper/2209.14046/) (2022)
- [Fine-grained Cross-modal Fusion based Refinement for Text-to-Image Synthesis](https://mlsanity.com/paper/2302.08706/) (2023)
- [DIFFNAT: Improving Diffusion Image Quality Using Natural Image Statistics](https://mlsanity.com/paper/2311.09753/) (2023)
- [DreamSalon: A Staged Diffusion Framework for Preserving Identity-Context in Editable Face Generation](https://mlsanity.com/paper/2403.19235/) (2024)
- [A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospects](https://mlsanity.com/paper/2506.19769/) (2025)
- [ALR-GAN: Adaptive Layout Refinement for Text-to-Image Synthesis](https://mlsanity.com/paper/2304.06297/) (2023)

## Abstract (arXiv)

Text-to-image synthesis refers to generating an image from a given text description, the key goal of which lies in photo realism and semantic consistency. Previous methods usually generate an initial image with sentence embedding and then refine it with fine-grained word embedding. Despite the significant progress, the 'aspect' information (e.g., red eyes) contained in the text, referring to several words rather than a word that depicts 'a particular part or feature of something', is often ignored, which is highly helpful for synthesizing image details. How to make better utilization of aspect information in text-to-image synthesis still remains an unresolved challenge. To address this problem, in this paper, we propose a Dynamic Aspect-awarE GAN (DAE-GAN) that represents text information comprehensively from multiple granularities, including sentence-level, word-level, and aspect-level. Moreover, inspired by human learning behaviors, we develop a novel Aspect-aware Dynamic Re-drawer (ADR) for image refinement, in which an Attended Global Refinement (AGR) module and an Aspect-aware Local Refinement (ALR) module are alternately employed. AGR utilizes word-level embedding to globally enhance the previously generated image, while ALR dynamically employs aspect-level embedding to refine image details from a local perspective. Finally, a corresponding matching loss function is designed to ensure the text-image semantic consistency at different levels. Extensive experiments on two well-studied and publicly available datasets (i.e., CUB-200 and COCO) demonstrate the superiority and rationality of our method.