mlsanityData to 14 Oct 2026

Probing the Creativity of Large Language Models: Can models produce divergent semantic association?

arXiv 2310.11158 ↗First posted 17 Oct 2023cs.CL cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

Large language models possess remarkable capacity for processing language, but it remains unclear whether these models can further generate creative content. The present study aims to investigate the creative thinking of large language models through a cognitive perspective. We utilize the divergent association task (DAT), an objective measurement of creativity that asks models to generate unrelated words and calculates the semantic distance between them. We compare the results across different models and decoding strategies. Our findings indicate that: (1) When using the greedy search strategy, GPT-4 outperforms 96% of humans, while GPT-3.5-turbo exceeds the average human level. (2) Stochastic sampling and temperature scaling are effective to obtain higher DAT scores for models except GPT-4, but face a trade-off between creativity and stability. These results imply that advanced large language models have divergent semantic associations, which is a fundamental process underlying creativity.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 23 of its 44 references that are in the map. First posted on arXiv 17 Oct 2023.

Builds on

23 of 44 references.

References · 23

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  8. 2018Hierarchical Neural Story Generationcited by 1,193
  9. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  10. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  11. 2022Using cognitive psychology to understand GPT-3cited by 398
  12. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
11 more
  1. 2023Challenges and Applications of Large Language Modelscited by 254
  2. 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
  3. 2020Trading Off Diversity and Quality in Natural Language Generationcited by 96
  4. 2023Boosting Theory-of-Mind Performance in Large Language Models via Promptingcited by 64
  5. 2020Word meaning in minds and machinescited by 51
  6. 2023Artificial muses: Generative Artificial Intelligence Chatbots Have Risen to Human-Level Creativitycited by 24
  7. 2023ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Modelscited by 24
  8. 2023Can ChatGPT be used to generate scientific hypotheses?cited by 21
  9. 2020Decoding Methods for Neural Narrative Generationcited by 18
  10. 2022On the probability-quality paradox in language generationcited by 16
  11. 2023In BLOOM: Creativity and Affinity in Artificial Lyrics and Artcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 17; 2026: 8.

33 papers.

Sort:

By area:

  1. 2025We're Different, We're the Same: Creative Homogeneity Across LLMscited by 42
  2. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 37
  3. 2024HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testingcited by 37
  4. 2024Large Language Models as Markov Chainscited by 35
  5. 2024Divergent Creativity in Humans and Large Language Modelscited by 31
  6. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 29
  7. 2024Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performancecited by 20
  8. 2024Large Language Models show both individual and collective creativity comparable to humanscited by 11
  9. 2025A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Modelscited by 10
  10. 2026Min-k Sampling: Decoupling Truncation from Temperature Scaling via Relative Logit Dynamicscited by 9
  11. 2024Creativity in AI: Progresses and Challengescited by 7
  12. 2025Measuring Combinatorial Creativity in Large Language Models and Humanscited by 7
21 more
  1. 2025Probing and Inducing Combinational Creativity in Vision-Language Modelscited by 6
  2. 2025GODBench: A Benchmark for Multimodal Large Language Models in Video Comment Artcited by 5
  3. 2025Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Riskscited by 5
  4. 2024What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworkscited by 4
  5. 2025MetaMuse: Algorithm Generation via Creative Ideationcited by 4
  6. 2025Attention to Non-Adopterscited by 4
  7. 2025DiffSampling: Enhancing Diversity and Accuracy in Neural Text Generationcited by 3
  8. 2026Beyond Divergent Creativity: A Human-Based Evaluation of Creativity in Large Language Modelscited by 3
  9. 2025Can Hallucinations Help? Boosting LLMs for Drug Discoverycited by 2
  10. 2026Automated Creativity Evaluation of Language Models Across Open-Ended Taskscited by 2
  11. 2025Defining and Quantifying Creative Behavior in Popular Image Generatorscited by 1
  12. 2025Evaluation Hallucination in Multi-Round Incomplete Information Lateral-Driven Reasoning Taskscited by 1
  13. 2025GuessBench: Sensemaking Multimodal Creativity in the Wildcited by 1
  14. 2025Leveraging Large Models to Evaluate Novel Content: A Case Study on Advertisement Creativitycited by 0
  15. 2025E.A.R.T.H.: Structuring Creative Evolution through Model Error in Generative AIcited by 0
  16. 2025AssoCiAm: A Benchmark for Evaluating Association Thinking while Circumventing Ambiguitycited by 0
  17. 2026Assessing the Creativity of Large Language Models: Testing, Limits, and New Frontierscited by 0
  18. 2026Where Models Converge and Humans Diverge: A Coverage Framework for Distributional Pluralism in Open-Ended Generationcited by 0
  19. 2026How LLMs Build Fictional Worlds: Setting and Narrative Space in AI-Generated Creative Storytellingcited by 0
  20. 2026Breaking Homogeneity: Diversifying Persona Sets for Creative LLM Outputscited by 0
  21. 2026Evaluating LLM Creativity through Long-Tail Performancecited by 0