# Predicting Human Similarity Judgments Using Large Language Models

arXiv: https://arxiv.org/abs/2202.04728 · HTML: https://mlsanity.com/paper/2202.04728/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 9 Feb 2022.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Human uncertainty makes classification more robust](https://mlsanity.com/paper/1908.07086/) (2019)
- [Word meaning in minds and machines](https://mlsanity.com/paper/2008.01766/) (2020)
- [Crisscrossed Captions: Extended Intramodal and Intermodal Semantic Similarity Judgments for MS-COCO](https://mlsanity.com/paper/2004.15020/) (2020)
- [Gibbs Sampling with People](https://mlsanity.com/paper/2008.02595/) (2020)
- [Extracting low-dimensional psychological representations from convolutional neural networks](https://mlsanity.com/paper/2005.14363/) (2020)
- [Generative timbre spaces: regularizing variational auto-encoders with perceptual metrics](https://mlsanity.com/paper/1805.08501/) (2018)
- [Capturing human categorization of natural images at scale by combining deep networks and cognitive models](https://mlsanity.com/paper/1904.12690/) (2019)

## Built on by (most cited first, 10 of 10)

- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Alignment with human representations supports robust few-shot learning](https://mlsanity.com/paper/2301.11990/) (2023)
- [Words are all you need? Language as an approximation for human similarity judgments](https://mlsanity.com/paper/2206.04105/) (2022)
- [Text Encoders Lack Knowledge: Leveraging Generative LLMs for Domain-Specific Semantic Textual Similarity](https://mlsanity.com/paper/2309.06541/) (2023)
- [Learning Human-like Representations to Enable Learning Human Values](https://mlsanity.com/paper/2312.14106/) (2023)
- [CoCoG: Controllable Visual Stimuli Generation based on Human Concept Representations](https://mlsanity.com/paper/2404.16482/) (2024)
- [Harnessing Collective Intelligence Under a Lack of Cultural Consensus](https://mlsanity.com/paper/2309.09787/) (2023)
- [Analyzing the Roles of Language and Vision in Learning from Limited Data](https://mlsanity.com/paper/2403.19669/) (2024)
- [AI-enhanced semantic feature norms for 786 concepts](https://mlsanity.com/paper/2505.10718/) (2025)
- [Uncovering the Computational Ingredients of Human-Like Representations in LLMs](https://mlsanity.com/paper/2510.01030/) (2025)

## Abstract (arXiv)

Similarity judgments provide a well-established method for accessing mental representations, with applications in psychology, neuroscience and machine learning. However, collecting similarity judgments can be prohibitively expensive for naturalistic datasets as the number of comparisons grows quadratically in the number of stimuli. One way to tackle this problem is to construct approximation procedures that rely on more accessible proxies for predicting similarity. Here we leverage recent advances in language models and online recruitment, proposing an efficient domain-general procedure for predicting human similarity judgments based on text descriptions. Intuitively, similar stimuli are likely to evoke similar descriptions, allowing us to use description similarity to predict pairwise similarity judgments. Crucially, the number of descriptions required grows only linearly with the number of stimuli, drastically reducing the amount of data required. We test this procedure on six datasets of naturalistic images and show that our models outperform previous approaches based on visual information.