# Quantifying the visual concreteness of words and topics in multimodal datasets

arXiv: https://arxiv.org/abs/1804.06786 · HTML: https://mlsanity.com/paper/1804.06786/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2018.

## Built on by (most cited first, 17 of 17)

- [Separating Skills and Concepts for Novel Visual Question Answering](https://mlsanity.com/paper/2107.09106/) (2021)
- [Billion-Scale Pretraining with Vision Transformers for Multi-Task Visual Representations](https://mlsanity.com/paper/2108.05887/) (2021)
- [Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understanding](https://mlsanity.com/paper/2303.12513/) (2023)
- [Mitigating Open-Vocabulary Caption Hallucinations](https://mlsanity.com/paper/2312.03631/) (2023)
- [PACS: A Dataset for Physical Audiovisual CommonSense Reasoning](https://mlsanity.com/paper/2203.11130/) (2022)
- [Revisiting Document Representations for Large-Scale Zero-Shot Learning](https://mlsanity.com/paper/2104.10355/) (2021)
- [Multi-Modal Framing Analysis of News](https://mlsanity.com/paper/2503.20960/) (2025)
- [Composition and Deformance: Measuring Imageability with a Text-to-Image Model](https://mlsanity.com/paper/2306.03168/) (2023)
- [Unveiling the Mystery of Visual Attributes of Concrete and Abstract Concepts: Variability, Nearest Neighbors, and Challenging Categories](https://mlsanity.com/paper/2410.11657/) (2024)
- [ICC: Quantifying Image Caption Concreteness for Multimodal Dataset Curation](https://mlsanity.com/paper/2403.01306/) (2024)
- [Understanding the Cognitive Complexity in Language Elicited by Product Images](https://mlsanity.com/paper/2409.16521/) (2024)
- [Uncovering Visual-Semantic Psycholinguistic Properties from the Distributional Structure of Text Embedding Space](https://mlsanity.com/paper/2505.23029/) (2025)
- [The impact of abstract and object tags on image privacy classification](https://mlsanity.com/paper/2510.07976/) (2025)
- [Grounded Concreteness: Human-Like Concreteness Sensitivity in Vision-Language Models](https://mlsanity.com/paper/2601.18065/) (2026)
- [Concrete Jungle: Towards Concreteness Paved Contrastive Negative Mining for Compositional Understanding](https://mlsanity.com/paper/2604.13313/) (2026)
- [Exploring Concreteness Through a Figurative Lens](https://mlsanity.com/paper/2604.18296/) (2026)
- [CREATE: Creativity Evaluation of VLMs using the Visual Remote Associates Test](https://mlsanity.com/paper/2610.18569/) (2026)

## Abstract (arXiv)

Multimodal machine learning algorithms aim to learn visual-textual correspondences. Previous work suggests that concepts with concrete visual manifestations may be easier to learn than concepts with abstract ones. We give an algorithm for automatically computing the visual concreteness of words and topics within multimodal datasets. We apply the approach in four settings, ranging from image captions to images/text scraped from historical books. In addition to enabling explorations of concepts in multimodal datasets, our concreteness scores predict the capacity of machine learning algorithms to learn textual/visual relationships. We find that 1) concrete concepts are indeed easier to learn; 2) the large number of algorithms we consider have similar failure cases; 3) the precise positive relationship between concreteness and performance varies between datasets. We conclude with recommendations for using concreteness scores to facilitate future multimodal research.