# CREATE: Testing LLMs for Associative Creativity

arXiv: https://arxiv.org/abs/2603.09970 · HTML: https://mlsanity.com/paper/2603.09970/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 51 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Builds on (most cited first, 27 of 27)

- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [A Roadmap to Pluralistic Alignment](https://mlsanity.com/paper/2402.05070/) (2024)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)](https://mlsanity.com/paper/2510.22954/) (2025)
- [Homogenization Effects of Large Language Models on Human Creative Ideation](https://mlsanity.com/paper/2402.01536/) (2024)
- [DiscoveryBench: Towards Data-Driven Discovery with Large Language Models](https://mlsanity.com/paper/2407.01725/) (2024)
- [Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity](https://mlsanity.com/paper/2510.01171/) (2025)
- [Kosmos: An AI Scientist for Autonomous Discovery](https://mlsanity.com/paper/2511.02824/) (2025)
- [The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas](https://mlsanity.com/paper/2506.20803/) (2025)
- [Base Models Beat Aligned Models at Randomness and Creativity](https://mlsanity.com/paper/2505.00047/) (2025)
- [Forcing Diffuse Distributions out of Language Models](https://mlsanity.com/paper/2404.10859/) (2024)
- [We're Different, We're the Same: Creative Homogeneity Across LLMs](https://mlsanity.com/paper/2501.19361/) (2025)
- [LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing](https://mlsanity.com/paper/2507.00769/) (2025)
- [Benchmarking Language Model Creativity: A Case Study on Code Generation](https://mlsanity.com/paper/2407.09007/) (2024)
- [HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generation](https://mlsanity.com/paper/2504.11524/) (2025)
- [From Distributional to Overton Pluralism: Investigating Large Language Model Alignment](https://mlsanity.com/paper/2406.17692/) (2024)
- [AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprise](https://mlsanity.com/paper/2507.00310/) (2025)
- [Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction](https://mlsanity.com/paper/2504.15266/) (2025)
- [Towards Execution-Grounded Automated AI Research](https://mlsanity.com/paper/2601.14525/) (2026)
- [IdeaBench: Benchmarking Large Language Models for Research Idea Generation](https://mlsanity.com/paper/2411.02429/) (2024)
- [MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge](https://mlsanity.com/paper/2412.17032/) (2024)
- [Knowledge Crosswords: Geometric Knowledge Reasoning with Large Language Models](https://mlsanity.com/paper/2310.01290/) (2023)
- [EvalAgent: Discovering Implicit Evaluation Criteria from the Web](https://mlsanity.com/paper/2504.15219/) (2025)
- [Measuring Combinatorial Creativity in Large Language Models and Humans](https://mlsanity.com/paper/2509.21043/) (2025)

## Built on by (most cited first, 7 of 7)

- [StoryScope: Investigating idiosyncrasies in AI fiction](https://mlsanity.com/paper/2604.03136/) (2026)
- [GIANTS: Generative Insight Anticipation from Scientific Literature](https://mlsanity.com/paper/2604.09793/) (2026)
- [From Reasoning Depth to Reasoning Breadth: Evaluating Multi-Point Associative Reasoning in Large Language Models](https://mlsanity.com/paper/2608.10444/) (2026)
- [CresOWLve: Benchmarking Creative Problem-Solving Over Real-World Knowledge](https://mlsanity.com/paper/2604.03374/) (2026)
- [Assessing the Creativity of Large Language Models: Testing, Limits, and New Frontiers](https://mlsanity.com/paper/2605.13450/) (2026)
- [UnpredictaBench: A Benchmark for Evaluating Distributional Randomness in LLMs](https://mlsanity.com/paper/2606.06622/) (2026)
- [GENIE: A Fine-Grained Measure for Novelty](https://mlsanity.com/paper/2606.12790/) (2026)

## Abstract (arXiv)

A key component of creativity is associative reasoning: the ability to draw novel yet meaningful connections between concepts. We introduce CREATE, a benchmark designed to evaluate models' capacity for creative associative reasoning. CREATE requires models to generate sets of paths connecting concepts in a model's parametric knowledge. Paths should have high specificity (distinctiveness and closeness of the concept connection) and high diversity (dissimilarity from other paths), and models are scored more highly if they produce a larger set of strong, diverse paths. This task shares demands of real creativity tasks like hypothesis generation, including an extremely large search space, but enables collection of a sizable benchmark with objective answer grading. Evaluation of frontier models shows that the strongest models achieve higher creative utility than others, with the high multiplicity of answers and complexity of the search making benchmark saturation difficult to achieve. Furthermore, our results illustrate that thinking models are not always more effective on our task, even with high token budgets. Recent approaches for creative prompting give some but limited additional improvement. CREATE provides a sandbox for developing new methods to improve models' capacity for associative creativity.