# CREATE: Creativity Evaluation of VLMs using the Visual Remote Associates Test

arXiv: https://arxiv.org/abs/2610.18569 · HTML: https://mlsanity.com/paper/2610.18569/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression](https://mlsanity.com/paper/2609.19969/) (2026)
- [PuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Abstract Visual Patterns](https://mlsanity.com/paper/2403.13315/) (2024)
- [Are Language Models Puzzle Prodigies? Algorithmic Puzzles Unveil Serious Challenges in Multimodal Reasoning](https://mlsanity.com/paper/2403.03864/) (2024)
- [Quantifying the visual concreteness of words and topics in multimodal datasets](https://mlsanity.com/paper/1804.06786/) (2018)
- [Predicting Concreteness and Imageability of Words Within and Across Languages via Word Embeddings](https://mlsanity.com/paper/1807.02903/) (2018)
- [Composition and Deformance: Measuring Imageability with a Text-to-Image Model](https://mlsanity.com/paper/2306.03168/) (2023)
- ["I See What You Did There": Can Large Vision-Language Models Understand Multimodal Puns?](https://mlsanity.com/paper/2604.05930/) (2026)

## Abstract (arXiv)

Vision language models (VLMs) are used in creative tasks such as visual storytelling, design ideation, art interpretation, and advertising. Yet we lack principled ways to measure the lateral reasoning of these models compared to human potential. We introduce CREATE, a semi-automatically constructed benchmark containing 155 visual Remote Associates Test problems. These problems require a solver to find an underlying connection among three seemingly dissimilar images. We conduct a human study (N=25) to validate and ground the quality of our benchmark, and evaluate 11 VLMs. Our findings show that most VLMs underperform humans on this task. Our experiments indicate that the associative ability of VLMs depends on the cue concepts provided in the text and does not universally extend to visual puzzles.