A Comparative Approach to Assessing Linguistic Creativity of Large Language Models and Humans
Not on the map. Why?
Abstract (arXiv)
The following paper introduces a general linguistic creativity test for humans and Large Language Models (LLMs). The test consists of various tasks aimed at assessing their ability to generate new original words and phrases based on word formation processes (derivation and compounding) and on metaphorical language use. We administered the test to 24 humans and to an equal number of LLMs, and we automatically evaluated their answers using OCSAI tool for three criteria: Originality, Elaboration, and Flexibility. The results show that LLMs not only outperformed humans in all the assessed criteria, but did better in six out of the eight test tasks. We then computed the uniqueness of the individual answers, which showed some minor differences between humans and LLMs. Finally, we performed a short manual analysis of the dataset, which revealed that humans are more inclined towards E(extending)-creativity, while LLMs favor F(ixed)-creativity.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 31 references that are in the map. First posted on arXiv 16 Jul 2025.
Builds on
6 of 31 references.
References · 6
- 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 172
- 2022FLUTE: Figurative Language Understanding through Textual Explanationscited by 79
- 2023On the Creativity of Large Language Modelscited by 57
- 2023I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphorscited by 31
- 2024Evaluating Morphological Compositional Generalization in Large Language Modelscited by 19
- 2023From chocolate bunny to chocolate crocodile: Do Language Models Understand Noun Compounds?cited by 12
Built on by
Citing papers per year: 2026: 1.
1 paper.