# Exploring Concreteness Through a Figurative Lens

arXiv: https://arxiv.org/abs/2604.18296 · HTML: https://mlsanity.com/paper/2604.18296/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 58 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding](https://mlsanity.com/paper/2103.06779/) (2021)
- [A Closer Look into Mixture-of-Experts in Large Language Models](https://mlsanity.com/paper/2406.18219/) (2024)
- [It's not Rocket Science : Interpreting Figurative Language in Narratives](https://mlsanity.com/paper/2109.00087/) (2021)
- [Metaphor Generation with Conceptual Mappings](https://mlsanity.com/paper/2106.01228/) (2021)
- [Metaphor Understanding Challenge Dataset for LLMs](https://mlsanity.com/paper/2403.11810/) (2024)
- [EPIE Dataset: A Corpus For Possible Idiomatic Expressions](https://mlsanity.com/paper/2006.09479/) (2020)
- [Quantifying the visual concreteness of words and topics in multimodal datasets](https://mlsanity.com/paper/1804.06786/) (2018)
- [ConMeC: A Dataset for Metonymy Resolution with Common Nouns](https://mlsanity.com/paper/2502.06087/) (2025)
- [A Computational Approach to Visual Metonymy](https://mlsanity.com/paper/2601.17706/) (2026)
- [Rhetorical Questions in LLM Representations: A Linear Probing Study](https://mlsanity.com/paper/2604.14128/) (2026)
- [The Representational Alignment between Humans and Language Models is implicitly driven by a Concreteness Effect](https://mlsanity.com/paper/2505.15682/) (2025)
- [MetFuse: Figurative Fusion between Metonymy and Metaphor](https://mlsanity.com/paper/2604.12919/) (2026)

## Abstract (arXiv)

Static concreteness ratings are widely used in NLP, yet a word's concreteness can shift with context, especially in figurative language such as metaphor, where common concrete nouns can take abstract interpretations. While such shifts are evident from context, it remains unclear how LLMs understand concreteness internally. We conduct a layer-wise and geometric analysis of LLM hidden representations across four model families, examining how models distinguish literal vs figurative uses of the same noun and how concreteness is organized in representation space. We find that LLMs separate literal and figurative usage in early layers, and that mid-to-late layers compress concreteness into a one-dimensional direction that is consistent across models. Finally, we show that this geometric structure is practically useful: a single concreteness direction supports efficient figurative-language classification and enables training-free steering of generation toward more literal or more figurative rewrites.