# Putting GPT-3's Creativity to the (Alternative Uses) Test

arXiv: https://arxiv.org/abs/2206.08932 · HTML: https://mlsanity.com/paper/2206.08932/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 24 references that are in the map.
- First posted on arXiv 10 Jun 2022.

## Builds on (most cited first, 2 of 2)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Abstraction and Analogy-Making in Artificial Intelligence](https://mlsanity.com/paper/2102.10717/) (2021)

## Built on by (most cited first, 30 of 62)

- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals](https://mlsanity.com/paper/2209.14958/) (2022)
- [Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models](https://mlsanity.com/paper/2307.14430/) (2023)
- [Who is GPT-3? An Exploration of Personality, Values and Demographics](https://mlsanity.com/paper/2209.14338/) (2022)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play](https://mlsanity.com/paper/2405.06373/) (2024)
- [On the Creativity of Large Language Models](https://mlsanity.com/paper/2304.00008/) (2023)
- [We're Different, We're the Same: Creative Homogeneity Across LLMs](https://mlsanity.com/paper/2501.19361/) (2025)
- [A Survey on Large Language Model Hallucination via a Creativity Perspective](https://mlsanity.com/paper/2402.06647/) (2024)
- [Assessing and Understanding Creativity in Large Language Models](https://mlsanity.com/paper/2401.12491/) (2024)
- [Divergent Creativity in Humans and Large Language Models](https://mlsanity.com/paper/2405.13012/) (2024)
- [AI for social science and social science of AI: A Survey](https://mlsanity.com/paper/2401.11839/) (2024)
- [Evaluating and Enhancing Large Language Models for Novelty Assessment in Scholarly Publications](https://mlsanity.com/paper/2409.16605/) (2024)
- [Creative Preference Optimization](https://mlsanity.com/paper/2505.14442/) (2025)
- [Machine Psychology](https://mlsanity.com/paper/2303.13988/) (2023)
- [How AI Ideas Affect the Creativity, Diversity, and Evolution of Human Ideas: Evidence From a Large, Dynamic Experiment](https://mlsanity.com/paper/2401.13481/) (2024)
- [Generative Transformers for Design Concept Generation](https://mlsanity.com/paper/2211.03468/) (2022)
- [Evaluating Creative Short Story Generation in Humans and Large Language Models](https://mlsanity.com/paper/2411.02316/) (2024)
- [Automated Creativity Evaluation for Large Language Models: A Reference-Based Approach](https://mlsanity.com/paper/2504.15784/) (2025)
- [CreativEval: Evaluating Creativity of LLM-Based Hardware Code Generation](https://mlsanity.com/paper/2404.08806/) (2024)
- [Susceptibility to Influence of Large Language Models](https://mlsanity.com/paper/2303.06074/) (2023)
- [Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review](https://mlsanity.com/paper/2401.01519/) (2024)
- [What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluation](https://mlsanity.com/paper/2408.14622/) (2024)
- [Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialogue](https://mlsanity.com/paper/2409.08330/) (2024)
- [Stick to your Role! Stability of Personal Values Expressed in Large Language Models](https://mlsanity.com/paper/2402.14846/) (2024)
- [Creative Beam Search: LLM-as-a-Judge For Improving Response Generation](https://mlsanity.com/paper/2405.00099/) (2024)
- [Do LLMs have Consistent Values?](https://mlsanity.com/paper/2407.12878/) (2024)
- [Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM](https://mlsanity.com/paper/2503.14478/) (2025)
- [Enhancing Creativity in Large Language Models through Associative Thinking Strategies](https://mlsanity.com/paper/2405.06715/) (2024)
- [An Evaluation of Estimative Uncertainty in Large Language Models](https://mlsanity.com/paper/2405.15185/) (2024)

## Abstract (arXiv)

AI large language models have (co-)produced amazing written works from newspaper articles to novels and poetry. These works meet the standards of the standard definition of creativity: being original and useful, and sometimes even the additional element of surprise. But can a large language model designed to predict the next text fragment provide creative, out-of-the-box, responses that still solve the problem at hand? We put Open AI's generative natural language model, GPT-3, to the test. Can it provide creative solutions to one of the most commonly used tests in creativity research? We assessed GPT-3's creativity on Guilford's Alternative Uses Test and compared its performance to previously collected human responses on expert ratings of originality, usefulness and surprise of responses, flexibility of each set of ideas as well as an automated method to measure creativity based on the semantic distance between a response and the AUT object in question. Our results show that -- on the whole -- humans currently outperform GPT-3 when it comes to creative output. But, we believe it is only a matter of time before GPT-3 catches up on this particular task. We discuss what this work reveals about human and AI creativity, creativity testing and our definition of creativity.