# Probing the Creativity of Large Language Models: Can models produce divergent semantic association?

arXiv: https://arxiv.org/abs/2310.11158 · HTML: https://mlsanity.com/paper/2310.11158/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [The Curse of Recursion: Training on Generated Data Makes Models Forget](https://mlsanity.com/paper/2305.17493/) (2023)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest](https://mlsanity.com/paper/2209.06293/) (2022)
- [Trading Off Diversity and Quality in Natural Language Generation](https://mlsanity.com/paper/2004.10450/) (2020)
- [Boosting Theory-of-Mind Performance in Large Language Models via Prompting](https://mlsanity.com/paper/2304.11490/) (2023)
- [Word meaning in minds and machines](https://mlsanity.com/paper/2008.01766/) (2020)
- [Artificial muses: Generative Artificial Intelligence Chatbots Have Risen to Human-Level Creativity](https://mlsanity.com/paper/2303.12003/) (2023)
- [ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models](https://mlsanity.com/paper/2306.04563/) (2023)
- [Can ChatGPT be used to generate scientific hypotheses?](https://mlsanity.com/paper/2304.12208/) (2023)
- [Decoding Methods for Neural Narrative Generation](https://mlsanity.com/paper/2010.07375/) (2020)
- [On the probability-quality paradox in language generation](https://mlsanity.com/paper/2203.17217/) (2022)
- [In BLOOM: Creativity and Affinity in Artificial Lyrics and Art](https://mlsanity.com/paper/2301.05402/) (2023)

## Built on by (most cited first, 30 of 33)

- [We're Different, We're the Same: Creative Homogeneity Across LLMs](https://mlsanity.com/paper/2501.19361/) (2025)
- [Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices](https://mlsanity.com/paper/2403.12503/) (2024)
- [HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testing](https://mlsanity.com/paper/2412.01778/) (2024)
- [Large Language Models as Markov Chains](https://mlsanity.com/paper/2410.02724/) (2024)
- [Divergent Creativity in Humans and Large Language Models](https://mlsanity.com/paper/2405.13012/) (2024)
- [Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction](https://mlsanity.com/paper/2504.15266/) (2025)
- [Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance](https://mlsanity.com/paper/2410.18889/) (2024)
- [Large Language Models show both individual and collective creativity comparable to humans](https://mlsanity.com/paper/2412.03151/) (2024)
- [A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models](https://mlsanity.com/paper/2501.15147/) (2025)
- [Min-k Sampling: Decoupling Truncation from Temperature Scaling via Relative Logit Dynamics](https://mlsanity.com/paper/2604.11012/) (2026)
- [Creativity in AI: Progresses and Challenges](https://mlsanity.com/paper/2410.17218/) (2024)
- [Measuring Combinatorial Creativity in Large Language Models and Humans](https://mlsanity.com/paper/2509.21043/) (2025)
- [Probing and Inducing Combinational Creativity in Vision-Language Models](https://mlsanity.com/paper/2504.13120/) (2025)
- [GODBench: A Benchmark for Multimodal Large Language Models in Video Comment Art](https://mlsanity.com/paper/2505.11436/) (2025)
- [Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Risks](https://mlsanity.com/paper/2505.19806/) (2025)
- [What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks](https://mlsanity.com/paper/2407.03007/) (2024)
- [MetaMuse: Algorithm Generation via Creative Ideation](https://mlsanity.com/paper/2510.03851/) (2025)
- [Attention to Non-Adopters](https://mlsanity.com/paper/2510.15951/) (2025)
- [DiffSampling: Enhancing Diversity and Accuracy in Neural Text Generation](https://mlsanity.com/paper/2502.14037/) (2025)
- [Beyond Divergent Creativity: A Human-Based Evaluation of Creativity in Large Language Models](https://mlsanity.com/paper/2601.20546/) (2026)
- [Can Hallucinations Help? Boosting LLMs for Drug Discovery](https://mlsanity.com/paper/2501.13824/) (2025)
- [Automated Creativity Evaluation of Language Models Across Open-Ended Tasks](https://mlsanity.com/paper/2606.11762/) (2026)
- [Defining and Quantifying Creative Behavior in Popular Image Generators](https://mlsanity.com/paper/2505.04497/) (2025)
- [Evaluation Hallucination in Multi-Round Incomplete Information Lateral-Driven Reasoning Tasks](https://mlsanity.com/paper/2505.23843/) (2025)
- [GuessBench: Sensemaking Multimodal Creativity in the Wild](https://mlsanity.com/paper/2506.00814/) (2025)
- [Leveraging Large Models to Evaluate Novel Content: A Case Study on Advertisement Creativity](https://mlsanity.com/paper/2503.00046/) (2025)
- [E.A.R.T.H.: Structuring Creative Evolution through Model Error in Generative AI](https://mlsanity.com/paper/2507.18004/) (2025)
- [AssoCiAm: A Benchmark for Evaluating Association Thinking while Circumventing Ambiguity](https://mlsanity.com/paper/2509.14171/) (2025)
- [Assessing the Creativity of Large Language Models: Testing, Limits, and New Frontiers](https://mlsanity.com/paper/2605.13450/) (2026)
- [Where Models Converge and Humans Diverge: A Coverage Framework for Distributional Pluralism in Open-Ended Generation](https://mlsanity.com/paper/2608.05576/) (2026)

## Abstract (arXiv)

Large language models possess remarkable capacity for processing language, but it remains unclear whether these models can further generate creative content. The present study aims to investigate the creative thinking of large language models through a cognitive perspective. We utilize the divergent association task (DAT), an objective measurement of creativity that asks models to generate unrelated words and calculates the semantic distance between them. We compare the results across different models and decoding strategies. Our findings indicate that: (1) When using the greedy search strategy, GPT-4 outperforms 96% of humans, while GPT-3.5-turbo exceeds the average human level. (2) Stochastic sampling and temperature scaling are effective to obtain higher DAT scores for models except GPT-4, but face a trade-off between creativity and stability. These results imply that advanced large language models have divergent semantic associations, which is a fundamental process underlying creativity.