# ManiGAN: Text-Guided Image Manipulation

arXiv: https://arxiv.org/abs/1912.06203 · HTML: https://mlsanity.com/paper/1912.06203/

## Key facts

- Cited by 156 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2019.

## Built on by (most cited first, 30 of 156)

- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://mlsanity.com/paper/2303.04671/) (2023)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing](https://mlsanity.com/paper/2306.10012/) (2023)
- [CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields](https://mlsanity.com/paper/2112.05139/) (2021)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Text2LIVE: Text-Driven Layered Image and Video Editing](https://mlsanity.com/paper/2204.02491/) (2022)
- [VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance](https://mlsanity.com/paper/2204.08583/) (2022)
- [Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting](https://mlsanity.com/paper/2212.06909/) (2022)
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [CLIPstyler: Image Style Transfer with a Single Text Condition](https://mlsanity.com/paper/2112.00374/) (2021)
- [Guiding Instruction-based Image Editing via Multimodal Large Language Models](https://mlsanity.com/paper/2309.17102/) (2023)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.08698/) (2022)
- [Collaborative Diffusion for Multi-Modal Face Generation and Editing](https://mlsanity.com/paper/2304.10530/) (2023)
- [Talk-to-Edit: Fine-Grained Facial Editing via Dialog](https://mlsanity.com/paper/2109.04425/) (2021)
- [EnvEdit: Environment Editing for Vision-and-Language Navigation](https://mlsanity.com/paper/2203.15685/) (2022)
- [Explainability of deep vision-based autonomous driving systems: Review and challenges](https://mlsanity.com/paper/2101.05307/) (2021)
- [ObjectFormer for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2203.14681/) (2022)
- [DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generation](https://mlsanity.com/paper/2305.03374/) (2023)
- [Towards Implicit Text-Guided 3D Shape Generation](https://mlsanity.com/paper/2203.14622/) (2022)
- [Shape-aware Text-driven Layered Video Editing](https://mlsanity.com/paper/2301.13173/) (2023)
- [Adversarial Text-to-Image Synthesis: A Review](https://mlsanity.com/paper/2101.09983/) (2021)
- [SinDiffusion: Learning a Diffusion Model from a Single Natural Image](https://mlsanity.com/paper/2211.12445/) (2022)
- [StyleMC: Multi-Channel Based Fast Text-Guided Image Generation and Manipulation](https://mlsanity.com/paper/2112.08493/) (2021)
- [Language-Driven Image Style Transfer](https://mlsanity.com/paper/2106.00178/) (2021)
- [DreamEdit: Subject-driven Image Editing](https://mlsanity.com/paper/2306.12624/) (2023)
- [M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformers](https://mlsanity.com/paper/2105.14211/) (2021)

## Abstract (arXiv)

The goal of our paper is to semantically edit parts of an image matching a given text that describes desired attributes (e.g., texture, colour, and background), while preserving other contents that are irrelevant to the text. To achieve this, we propose a novel generative adversarial network (ManiGAN), which contains two key components: text-image affine combination module (ACM) and detail correction module (DCM). The ACM selects image regions relevant to the given text and then correlates the regions with corresponding semantic words for effective manipulation. Meanwhile, it encodes original image features to help reconstruct text-irrelevant contents. The DCM rectifies mismatched attributes and completes missing contents of the synthetic image. Finally, we suggest a new metric for evaluating image manipulation results, in terms of both the generation of new attributes and the reconstruction of text-irrelevant contents. Extensive experiments on the CUB and COCO datasets demonstrate the superior performance of the proposed method. Code is available at https://github.com/mrlibw/ManiGAN.