# Dual Adversarial Inference for Text-to-Image Synthesis

arXiv: https://arxiv.org/abs/1908.05324 · HTML: https://mlsanity.com/paper/1908.05324/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2019.

## Built on by (most cited first, 17 of 17)

- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Cross-Modal Contrastive Learning for Text-to-Image Generation](https://mlsanity.com/paper/2101.04702/) (2021)
- [DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models](https://mlsanity.com/paper/2210.06998/) (2022)
- [Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2304.01816/) (2023)
- [Improved Vector Quantized Diffusion Models](https://mlsanity.com/paper/2205.16007/) (2022)
- [Adversarial Text-to-Image Synthesis: A Review](https://mlsanity.com/paper/2101.09983/) (2021)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [Learning Disentangled Representations in the Imaging Domain](https://mlsanity.com/paper/2108.12043/) (2021)
- [DiVAE: Photorealistic Images Synthesis with Denoising Diffusion Decoder](https://mlsanity.com/paper/2206.00386/) (2022)
- [Bi-LORA: A Vision-Language Approach for Synthetic Image Detection](https://mlsanity.com/paper/2404.01959/) (2024)
- [ARMANI: Part-level Garment-Text Alignment for Unified Cross-Modal Fashion Design](https://mlsanity.com/paper/2208.05621/) (2022)
- [DiffCloth: Diffusion Based Garment Synthesis and Manipulation via Structural Cross-modal Semantic Alignment](https://mlsanity.com/paper/2308.11206/) (2023)
- [Layout-Bridging Text-to-Image Synthesis](https://mlsanity.com/paper/2208.06162/) (2022)
- [CT-SGAN: Computed Tomography Synthesis GAN](https://mlsanity.com/paper/2110.09288/) (2021)
- [Each Attribute Matters: Contrastive Attention for Sentence-based Image Editing](https://mlsanity.com/paper/2110.11159/) (2021)
- [Self-Supervised Image-to-Text and Text-to-Image Synthesis](https://mlsanity.com/paper/2112.04928/) (2021)
- [on the effectiveness of generative adversarial network on anomaly detection](https://mlsanity.com/paper/2112.15541/) (2021)

## Abstract (arXiv)

Synthesizing images from a given text description involves engaging two types of information: the content, which includes information explicitly described in the text (e.g., color, composition, etc.), and the style, which is usually not well described in the text (e.g., location, quantity, size, etc.). However, in previous works, it is typically treated as a process of generating images only from the content, i.e., without considering learning meaningful style representations. In this paper, we aim to learn two variables that are disentangled in the latent space, representing content and style respectively. We achieve this by augmenting current text-to-image synthesis frameworks with a dual adversarial inference mechanism. Through extensive experiments, we show that our model learns, in an unsupervised manner, style representations corresponding to certain meaningful information present in the image that are not well described in the text. The new framework also improves the quality of synthesized images when evaluated on Oxford-102, CUB and COCO datasets.