# GenColor: Generative Color-Concept Association in Visual Design

arXiv: https://arxiv.org/abs/2503.03236 · HTML: https://mlsanity.com/paper/2503.03236/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 85 references that are in the map.
- First posted on arXiv 5 Mar 2025.

## Builds on (most cited first, 26 of 26)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [Fake it till you make it: Learning transferable representations from synthetic ImageNet clones](https://mlsanity.com/paper/2212.08420/) (2022)
- [Social Biases through the Text-to-Image Generation Lens](https://mlsanity.com/paper/2304.06034/) (2023)
- [Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentation](https://mlsanity.com/paper/2309.14303/) (2023)
- [The Creativity of Text-to-Image Generation](https://mlsanity.com/paper/2206.02904/) (2022)
- [Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation](https://mlsanity.com/paper/2404.01030/) (2024)
- [Investigating Prompt Engineering in Diffusion Models](https://mlsanity.com/paper/2211.15462/) (2022)
- [Understanding the Impact of Negative Prompts: When and How Do They Take Effect?](https://mlsanity.com/paper/2406.02965/) (2024)
- [SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matching](https://mlsanity.com/paper/2310.17569/) (2023)
- [Identifying Mislabeled Instances in Classification Datasets](https://mlsanity.com/paper/1912.05283/) (2019)
- [From Melting Pots to Misrepresentations: Exploring Harms in Generative AI](https://mlsanity.com/paper/2403.10776/) (2024)
- [TypeDance: Creating Semantic Typographic Logos from Image through Personalized Generation](https://mlsanity.com/paper/2401.11094/) (2024)
- [What's in a Decade? Transforming Faces Through Time](https://mlsanity.com/paper/2210.06642/) (2022)
- [C2Ideas: Supporting Creative Interior Color Design Ideation with Large Language Model](https://mlsanity.com/paper/2401.12586/) (2024)
- [Colors -Messengers of Concepts: Visual Design Mining for Learning Color Semantics](https://mlsanity.com/paper/1505.06532/) (2015)
- [Context Matters: A Theory of Semantic Discriminability for Perceptual Encoding Systems](https://mlsanity.com/paper/2108.03685/) (2021)
- [Estimating Color-Concept Associations from Image Statistics](https://mlsanity.com/paper/1908.00220/) (2019)

## Built on by (most cited first, 3 of 3)

- [Color Me Correctly: Bridging Perceptual Color Spaces and Text Embeddings for Improved Diffusion Generation](https://mlsanity.com/paper/2509.10058/) (2025)
- [NumColor: Precise Numeric Color Control in Text-to-Image Generation](https://mlsanity.com/paper/2603.13547/) (2026)
- [Recolour What Matters: Region-Aware Colour Editing via Token-Level Diffusion](https://mlsanity.com/paper/2603.18466/) (2026)

## Abstract (arXiv)

Existing approaches for color-concept association typically rely on query-based image referencing, and color extraction from image references. However, these approaches are effective only for common concepts, and are vulnerable to unstable image referencing and varying image conditions. Our formative study with designers underscores the need for primary-accent color compositions and context-dependent colors (e.g., 'clear' vs. 'polluted' sky) in design. In response, we introduce a generative approach for mining semantically resonant colors leveraging images generated by text-to-image models. Our insight is that contemporary text-to-image models can resemble visual patterns from large-scale real-world data. The framework comprises three stages: concept instancing produces generative samples using diffusion models, text-guided image segmentation identifies concept-relevant regions within the image, and color association extracts primarily accompanied by accent colors. Quantitative comparisons with expert designs validate our approach's effectiveness, and we demonstrate the applicability through cases in various design scenarios and a gallery.