# Composition and Deformance: Measuring Imageability with a Text-to-Image Model

arXiv: https://arxiv.org/abs/2306.03168 · HTML: https://mlsanity.com/paper/2306.03168/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 24 references that are in the map.
- First posted on arXiv 5 Jun 2023.

## Builds on (most cited first, 4 of 4)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://mlsanity.com/paper/1804.11283/) (2018)
- [Quantifying the visual concreteness of words and topics in multimodal datasets](https://mlsanity.com/paper/1804.06786/) (2018)

## Built on by (most cited first, 5 of 5)

- [Mitigating Open-Vocabulary Caption Hallucinations](https://mlsanity.com/paper/2312.03631/) (2023)
- [Exploring Automated Keyword Mnemonics Generation with Large Language Models via Overgenerate-and-Rank](https://mlsanity.com/paper/2409.13952/) (2024)
- [ICC: Quantifying Image Caption Concreteness for Multimodal Dataset Curation](https://mlsanity.com/paper/2403.01306/) (2024)
- [Uncovering Visual-Semantic Psycholinguistic Properties from the Distributional Structure of Text Embedding Space](https://mlsanity.com/paper/2505.23029/) (2025)
- [CREATE: Creativity Evaluation of VLMs using the Visual Remote Associates Test](https://mlsanity.com/paper/2610.18569/) (2026)

## Abstract (arXiv)

Although psycholinguists and psychologists have long studied the tendency of linguistic strings to evoke mental images in hearers or readers, most computational studies have applied this concept of imageability only to isolated words. Using recent developments in text-to-image generation models, such as DALLE mini, we propose computational methods that use generated images to measure the imageability of both single English words and connected text. We sample text prompts for image generation from three corpora: human-generated image captions, news article sentences, and poem lines. We subject these prompts to different deformances to examine the model's ability to detect changes in imageability caused by compositional change. We find high correlation between the proposed computational measures of imageability and human judgments of individual words. We also find the proposed measures more consistently respond to changes in compositionality than baseline approaches. We discuss possible effects of model training and implications for the study of compositionality in text-to-image models.