# MirrorGAN: Learning Text-to-image Generation by Redescription

arXiv: https://arxiv.org/abs/1903.05854 · HTML: https://mlsanity.com/paper/1903.05854/

## Key facts

- Cited by 176 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2019.

## Built on by (most cited first, 30 of 176)

- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators](https://mlsanity.com/paper/2303.13439/) (2023)
- [Cross-Modal Contrastive Learning for Text-to-Image Generation](https://mlsanity.com/paper/2101.04702/) (2021)
- [GODIVA: Generating Open-DomaIn Videos from nAtural Descriptions](https://mlsanity.com/paper/2104.14806/) (2021)
- [TextDiffuser: Diffusion Models as Text Painters](https://mlsanity.com/paper/2305.10855/) (2023)
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.14704/) (2022)
- [Cache Me if You Can: Accelerating Diffusion Models through Block Caching](https://mlsanity.com/paper/2312.03209/) (2023)
- [Rich Human Feedback for Text-to-Image Generation](https://mlsanity.com/paper/2312.10240/) (2023)
- [PatchCraft: Exploring Texture Patch for Efficient AI-generated Image Detection](https://mlsanity.com/paper/2311.12397/) (2023)
- [Vision-Language Pre-training: Basics, Recent Advances, and Future Trends](https://mlsanity.com/paper/2210.09263/) (2022)
- [Paint by Word](https://mlsanity.com/paper/2103.10951/) (2021)
- [Prompt Tuning Inversion for Text-Driven Image Editing Using Diffusion Models](https://mlsanity.com/paper/2305.04441/) (2023)
- [GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesis](https://mlsanity.com/paper/2301.12959/) (2023)
- [CLIP-Driven Fine-grained Text-Image Person Re-identification](https://mlsanity.com/paper/2210.10276/) (2022)
- [IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformers](https://mlsanity.com/paper/2304.14400/) (2023)
- [StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation](https://mlsanity.com/paper/2209.06192/) (2022)
- [Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models](https://mlsanity.com/paper/2211.10950/) (2022)
- [Towards Implicit Text-Guided 3D Shape Generation](https://mlsanity.com/paper/2203.14622/) (2022)
- [Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2304.01816/) (2023)
- [Synthetic Data -- what, why and how?](https://mlsanity.com/paper/2205.03257/) (2022)
- [Improving Text-to-Image Synthesis Using Contrastive Learning](https://mlsanity.com/paper/2107.02423/) (2021)
- [CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP](https://mlsanity.com/paper/2203.00386/) (2022)
- [Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation](https://mlsanity.com/paper/2303.09319/) (2023)
- [Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models](https://mlsanity.com/paper/2407.02482/) (2024)
- [Text to Image Generation with Semantic-Spatial Aware GAN](https://mlsanity.com/paper/2104.00567/) (2021)
- [Adversarial Text-to-Image Synthesis: A Review](https://mlsanity.com/paper/2101.09983/) (2021)

## Abstract (arXiv)

Generating an image from a given text description has two goals: visual realism and semantic consistency. Although significant progress has been made in generating high-quality and visually realistic images using generative adversarial networks, guaranteeing semantic consistency between the text description and visual content remains very challenging. In this paper, we address this problem by proposing a novel global-local attentive and semantic-preserving text-to-image-to-text framework called MirrorGAN. MirrorGAN exploits the idea of learning text-to-image generation by redescription and consists of three modules: a semantic text embedding module (STEM), a global-local collaborative attentive module for cascaded image generation (GLAM), and a semantic text regeneration and alignment module (STREAM). STEM generates word- and sentence-level embeddings. GLAM has a cascaded architecture for generating target images from coarse to fine scales, leveraging both local word attention and global sentence attention to progressively enhance the diversity and semantic consistency of the generated images. STREAM seeks to regenerate the text description from the generated image, which semantically aligns with the given text description. Thorough experiments on two public benchmark datasets demonstrate the superiority of MirrorGAN over other representative state-of-the-art methods.