mlsanityData to 14 Oct 2026

Each Attribute Matters: Contrastive Attention for Sentence-based Image Editing

arXiv 2110.11159 ↗First posted 21 Oct 2021cs.CV cs.LG cs.MMcited by 1

Not on the map. Why?

Abstract (arXiv)

Sentence-based Image Editing (SIE) aims to deploy natural language to edit an image. Offering potentials to reduce expensive manual editing, SIE has attracted much interest recently. However, existing methods can hardly produce accurate editing and even lead to failures in attribute editing when the query sentence is with multiple editable attributes. To cope with this problem, by focusing on enhancing the difference between attributes, this paper proposes a novel model called Contrastive Attention Generative Adversarial Network (CA-GAN), which is inspired from contrastive training. Specifically, we first design a novel contrastive attention module to enlarge the editing difference between random combinations of attributes which are formed during training. We then construct an attribute discriminator to ensure effective editing on each attribute. A series of experiments show that our method can generate very encouraging results in sentence-based image editing with multiple attributes on CUB and COCO dataset. Our code is available at https://github.com/Zlq2021/CA-GAN

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 21 Oct 2021.

Builds on

32 of 49 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  9. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  10. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  11. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 639
  12. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
20 more
  1. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  2. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  3. 2020Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learningcited by 220
  4. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  5. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  6. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  7. 2020An Unsupervised Sentence Embedding Method by Mutual Information Maximizationcited by 91
  8. 2018Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Languagecited by 90
  9. 2017Semantic Image Synthesis via Adversarial Learningcited by 87
  10. 2018Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Networkcited by 64
  11. 2019Semantics Disentangling for Text-to-Image Generationcited by 61
  12. 2020InfoMax-GAN: Improved Adversarial Image Generation via Information Maximization and Contrastive Learningcited by 37
  13. 2019Multi-mapping Image-to-Image Translation via Learning Disentanglementcited by 35
  14. 2020Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructionscited by 25
  15. 2020Multi-Domain Multi-Task Rehearsal for Lifelong Learningcited by 24
  16. 2019Dual Adversarial Inference for Text-to-Image Synthesiscited by 17
  17. 2018Guide Me: Interacting with Deep Networkscited by 16
  18. 2020Emerging Disentanglement in Auto-Encoder Based Unsupervised Image Content Transfercited by 15
  19. 2018SingleGAN: Image-to-Image Translation by a Single-Generator Network using Multiple Generative Adversarial Learningcited by 7
  20. 2021Phraseformer: Multimodal Key-phrase Extraction using Transformer and Graph Embeddingcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022AGCN: Augmented Graph Convolutional Network for Lifelong Multi-label Image Recognitioncited by 16