# Generating Compositional Color Representations from Text

arXiv: https://arxiv.org/abs/2109.10477 · HTML: https://mlsanity.com/paper/2109.10477/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 58 references that are in the map.
- First posted on arXiv 22 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [CERT: Contrastive Self-supervised Learning for Language Understanding](https://mlsanity.com/paper/2005.12766/) (2020)
- [Task-Driven Modular Networks for Zero-Shot Compositional Learning](https://mlsanity.com/paper/1905.05908/) (2019)
- [Symmetry and Group in Attribute-Object Compositions](https://mlsanity.com/paper/2004.00587/) (2020)
- [Learn To Pay Attention](https://mlsanity.com/paper/1804.02391/) (2018)
- [Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding](https://mlsanity.com/paper/1703.10186/) (2017)
- [Language-Based Image Editing with Recurrent Attentive Models](https://mlsanity.com/paper/1711.06288/) (2017)
- [Learning Compositional Representations for Few-Shot Recognition](https://mlsanity.com/paper/1812.09213/) (2018)
- [Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation](https://mlsanity.com/paper/1804.04128/) (2018)
- [Attributes as Operators: Factorizing Unseen Attribute-Object Compositions](https://mlsanity.com/paper/1803.09851/) (2018)
- [Learning to Color from Language](https://mlsanity.com/paper/1804.06026/) (2018)
- [Task-Aware Feature Generation for Zero-Shot Compositional Learning](https://mlsanity.com/paper/1906.04854/) (2019)
- [Character Sequence Models for ColorfulWords](https://mlsanity.com/paper/1609.08777/) (2016)
- [Scene Graph Embeddings Using Relative Similarity Supervision](https://mlsanity.com/paper/2104.02381/) (2021)
- [Learning to Generate Compositional Color Descriptions](https://mlsanity.com/paper/1606.03821/) (2016)
- [Learning Colour Representations of Search Queries](https://mlsanity.com/paper/2006.09904/) (2020)

## Built on by (most cited first, 3 of 3)

- [Multimodal Color Recommendation in Vector Graphic Documents](https://mlsanity.com/paper/2308.04118/) (2023)
- [From Fragment to One Piece: A Survey on AI-Driven Graphic Design](https://mlsanity.com/paper/2503.18641/) (2025)
- [ColorGPT: Leveraging Large Language Models for Multimodal Color Recommendation](https://mlsanity.com/paper/2508.08987/) (2025)

## Abstract (arXiv)

We consider the cross-modal task of producing color representations for text phrases. Motivated by the fact that a significant fraction of user queries on an image search engine follow an (attribute, object) structure, we propose a generative adversarial network that generates color profiles for such bigrams. We design our pipeline to learn composition - the ability to combine seen attributes and objects to unseen pairs. We propose a novel dataset curation pipeline from existing public sources. We describe how a set of phrases of interest can be compiled using a graph propagation technique, and then mapped to images. While this dataset is specialized for our investigations on color, the method can be extended to other visual dimensions where composition is of interest. We provide detailed ablation studies that test the behavior of our GAN architecture with loss functions from the contrastive learning literature. We show that the generative model achieves lower Frechet Inception Distance than discriminative ones, and therefore predicts color profiles that better match those from real images. Finally, we demonstrate improved performance in image retrieval and classification, indicating the crucial role that color plays in these downstream tasks.