# Measuring Combinatorial Creativity in Large Language Models and Humans

arXiv: https://arxiv.org/abs/2509.21043 · HTML: https://mlsanity.com/paper/2509.21043/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 90 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Builds on (most cited first, 27 of 27)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [Multilingual E5 Text Embeddings: A Technical Report](https://mlsanity.com/paper/2402.05672/) (2024)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [Compositionality decomposed: how do neural networks generalise?](https://mlsanity.com/paper/1908.08351/) (2019)
- [COGS: A Compositional Generalization Challenge Based on Semantic Interpretation](https://mlsanity.com/paper/2010.05465/) (2020)
- [Abstraction and Analogy-Making in Artificial Intelligence](https://mlsanity.com/paper/2102.10717/) (2021)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [NoveltyBench: Evaluating Language Models for Humanlike Diversity](https://mlsanity.com/paper/2504.05228/) (2025)
- [The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas](https://mlsanity.com/paper/2506.20803/) (2025)
- [Probing the Creativity of Large Language Models: Can models produce divergent semantic association?](https://mlsanity.com/paper/2310.11158/) (2023)
- [Divergent Creativity in Humans and Large Language Models](https://mlsanity.com/paper/2405.13012/) (2024)
- [Evaluating the Robustness of Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2411.14215/) (2024)
- [Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction](https://mlsanity.com/paper/2504.15266/) (2025)
- [E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning](https://mlsanity.com/paper/2203.08480/) (2022)
- [PopBlends: Strategies for Conceptual Blending with Large Language Models](https://mlsanity.com/paper/2111.04920/) (2021)
- [Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks](https://mlsanity.com/paper/1711.00350/) (2017)
- [LLMs can Realize Combinatorial Creativity: Generating Creative Ideas via LLMs for Scientific Research](https://mlsanity.com/paper/2412.14141/) (2024)
- [A Survey on Compositional Learning of AI Models: Theoretical and Experimental Practices](https://mlsanity.com/paper/2406.08787/) (2024)
- [ANALOGICAL -- A Novel Benchmark for Long Text Analogy Evaluation in Large Language Models](https://mlsanity.com/paper/2305.05050/) (2023)
- [Spark: A System for Scientifically Creative Idea Generation](https://mlsanity.com/paper/2504.20090/) (2025)
- [Transformational Creativity in Science: A Graphical Theory](https://mlsanity.com/paper/2504.18687/) (2025)
- [Explaining Creative Artifacts](https://mlsanity.com/paper/2010.07126/) (2020)
- [Cooking Up Creativity: Enhancing LLM Creativity through Structured Recombination](https://mlsanity.com/paper/2504.20643/) (2025)
- [Modelling Analogies and Analogical Reasoning: Connecting Cognitive Science Theory and NLP Research](https://mlsanity.com/paper/2509.09381/) (2025)
- [Is a Peeled Apple Still Red? Evaluating LLMs' Ability for Conceptual Combination with Property Type](https://mlsanity.com/paper/2502.06086/) (2025)
- [Generalization and the Rise of System-level Creativity in Science](https://mlsanity.com/paper/2510.03240/) (2025)

## Built on by (most cited first, 7 of 7)

- [CREATE: Testing LLMs for Associative Creativity](https://mlsanity.com/paper/2603.09970/) (2026)
- [MoRI: Learning Motivation-Grounded Reasoning for Scientific Ideation in Large Language Models](https://mlsanity.com/paper/2603.19044/) (2026)
- [Beyond Fixed Representations: The Vocabulary and Verifier Gaps in Open-Ended AI](https://mlsanity.com/paper/2607.09560/) (2026)
- [Universe of Thoughts: Enabling Creative Reasoning with Large Language Models](https://mlsanity.com/paper/2511.20471/) (2025)
- [Assessing the Creativity of Large Language Models: Testing, Limits, and New Frontiers](https://mlsanity.com/paper/2605.13450/) (2026)
- [GENIE: A Fine-Grained Measure for Novelty](https://mlsanity.com/paper/2606.12790/) (2026)
- [Can LLMs Imagine Moral Alternatives Beyond Binary Dilemmas?](https://mlsanity.com/paper/2606.31213/) (2026)

## Abstract (arXiv)

Combinatorial creativity, the process of combining familiar ideas in unfamiliar ways, is one of the foremost engines of scientific, technological, and artistic progress. This work introduces Kombine, an LLM benchmark for combinatorial creativity tasking models to discover surprising, original, and useful associations, analogies, and blends between unrelated entities, and to use those analogies and blends to invent novel concepts by re-combining the parts of existing ones. In a large-scale study across 35 LLMs, we find that: (1) LLMs are 10-16 percentage points better at discovering associations and analogies than blends, and (2) exhibit an abstraction bottleneck on blending, in which even frontier LLMs struggle to find a shared abstraction between inputs and instantiate a coherent blended space. (3) LLMs also exhibit inventive homogeneity, in which one in five concepts invented for a given item reuses properties invented by another model. Moreover, (4) models are 14× more likely to reuse other invented properties on blends than analogies, and (5) models from the same provider are 2.3× more likely to reuse properties than those from different providers. Next, in a comparative study involving 75 humans on the same tasks, we find that (6) humans score below nearly every LLM on each task and (7) both humans and LLMs find blending most difficult, but association is easier than analogy for LLMs, while analogy is easier than association for humans. Lastly, (8) humans and LLMs access distinct regions of semantic space on each task, suggesting that combinatorial creativity could benefit from systematic human-AI collaboration to enhance conceptual diversity. In summary, we provide a framework to assess progress towards LLMs, AI systems, and human-AI collaborations more broadly, that can elicit combinatorial creativity in a setting relevant to scientific and technological innovation.