# Perceptual Structure in the Absence of Grounding for LLMs: The Impact of Abstractedness and Subjectivity in Color Language

arXiv: https://arxiv.org/abs/2311.13105 · HTML: https://mlsanity.com/paper/2311.13105/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 22 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/1808.09031/) (2018)
- [Assessing BERT's Syntactic Abilities](https://mlsanity.com/paper/1901.05287/) (2019)
- [Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color](https://mlsanity.com/paper/2109.06129/) (2021)
- [Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding](https://mlsanity.com/paper/1703.10186/) (2017)
- [Color naming reflects both perceptual structure and communicative need](https://mlsanity.com/paper/1805.06165/) (2018)
- [Character Sequence Models for ColorfulWords](https://mlsanity.com/paper/1609.08777/) (2016)
- [Generating Bilingual Pragmatic Color References](https://mlsanity.com/paper/1803.03917/) (2018)
- [Learning to Generate Compositional Color Descriptions](https://mlsanity.com/paper/1606.03821/) (2016)
- [Grounding learning of modifier dynamics: An application to color naming](https://mlsanity.com/paper/1909.07586/) (2019)

## Built on by (most cited first, 4 of 4)

- [Constructive Approach to Bidirectional Influence between Qualia Structure and Language Emergence](https://mlsanity.com/paper/2409.09413/) (2024)
- [Reflectance Estimation for Proximity Sensing by Vision-Language Models: Utilizing Distributional Semantics for Low-Level Cognition in Robotics](https://mlsanity.com/paper/2404.07717/) (2024)
- [Color Bind: Exploring Color Perception in Text-to-Image Models](https://mlsanity.com/paper/2508.19791/) (2025)
- [ColorGPT: Leveraging Large Language Models for Multimodal Color Recommendation](https://mlsanity.com/paper/2508.08987/) (2025)

## Abstract (arXiv)

The need for grounding in language understanding is an active research topic. Previous work has suggested that color perception and color language appear as a suitable test bed to empirically study the problem, given its cognitive significance and showing that there is considerable alignment between a defined color space and the feature space defined by a language model. To further study this issue, we collect a large scale source of colors and their descriptions, containing almost a 1 million examples , and perform an empirical analysis to compare two kinds of alignments: (i) inter-space, by learning a mapping between embedding space and color space, and (ii) intra-space, by means of prompting comparatives between color descriptions. Our results show that while color space alignment holds for monolexemic, highly pragmatic color descriptions, this alignment drops considerably in the presence of examples that exhibit elements of real linguistic usage such as subjectivity and abstractedness, suggesting that grounding may be required in such cases.