# Image-to-Image Translation with Text Guidance

arXiv: https://arxiv.org/abs/2002.05235 · HTML: https://mlsanity.com/paper/2002.05235/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2020.

## Built on by (most cited first, 10 of 10)

- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [ReCo: Region-Controlled Text-to-Image Generation](https://mlsanity.com/paper/2211.15518/) (2022)
- [SceneComposer: Any-Level Semantic Image Synthesis](https://mlsanity.com/paper/2211.11742/) (2022)
- [Word-Level Fine-Grained Story Visualization](https://mlsanity.com/paper/2208.02341/) (2022)
- [EKILA: Synthetic Media Provenance and Attribution for Generative Art](https://mlsanity.com/paper/2304.04639/) (2023)
- [Memory-Driven Text-to-Image Generation](https://mlsanity.com/paper/2208.07022/) (2022)
- [DT2I: Dense Text-to-Image Generation from Region Descriptions](https://mlsanity.com/paper/2204.02035/) (2022)
- [Aerial Diffusion: Text Guided Ground-to-Aerial View Translation from a Single Image using Diffusion Models](https://mlsanity.com/paper/2303.11444/) (2023)
- [Learning to Model Multimodal Semantic Alignment for Story Visualization](https://mlsanity.com/paper/2211.07289/) (2022)
- [Lightweight Long-Range Generative Adversarial Networks](https://mlsanity.com/paper/2209.03793/) (2022)

## Abstract (arXiv)

The goal of this paper is to embed controllable factors, i.e., natural language descriptions, into image-to-image translation with generative adversarial networks, which allows text descriptions to determine the visual attributes of synthetic images. We propose four key components: (1) the implementation of part-of-speech tagging to filter out non-semantic words in the given description, (2) the adoption of an affine combination module to effectively fuse different modality text and image features, (3) a novel refined multi-stage architecture to strengthen the differential ability of discriminators and the rectification ability of generators, and (4) a new structure loss to further improve discriminators to better distinguish real and synthetic images. Extensive experiments on the COCO dataset demonstrate that our method has a superior performance on both visual realism and semantic consistency with given descriptions.