mlsanityData to 7 Oct 2026

On the Creativity of Large Language Models

arXiv 2304.00008 ↗First posted 27 Mar 2023cs.AI cs.CL cs.CYcited by 57

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are revolutionizing several areas of Artificial Intelligence. One of the most remarkable applications is creative writing, e.g., poetry or storytelling: the generated outputs are often of astonishing quality. However, a natural question arises: can LLMs be really considered creative? In this article, we first analyze the development of LLMs under the lens of creativity theories, investigating the key open questions and challenges. In particular, we focus our discussion on the dimensions of value, novelty, and surprise as proposed by Margaret Boden in her work. Then, we consider different classic perspectives, namely product, process, press, and person. We discuss a set of ``easy'' and ``hard'' problems in machine creativity, presenting them in relation to LLMs. Finally, we examine the societal impact of these technologies with a particular focus on the creative industries, analyzing the opportunities offered, the challenges arising from them, and the potential associated risks, from both legal and ethical points of view.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Builds on 51 of its 151 references that are in the map. First posted on arXiv 27 Mar 2023.

Builds on

51 of 151 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2020Learning to summarize from human feedbackcited by 2,562
38 more
  1. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  2. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  3. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  4. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  5. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  6. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  7. 2017Continual Learning with Deep Generative Replaycited by 1,152
  8. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  9. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  10. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  11. 2022A Survey on In-context Learningcited by 717
  12. 2023A Watermark for Large Language Modelscited by 667
  13. 2015Generating Sentences from a Continuous Spacecited by 607
  14. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  15. 2022Using cognitive psychology to understand GPT-3cited by 389
  16. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  17. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  18. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 217
  19. 2022Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionalscited by 191
  20. 2022Talking About Large Language Modelscited by 174
  21. 2021Understanding the Capabilities, Limitations, and Societal Impact of Large Language Modelscited by 159
  22. 2019LAMOL: LAnguage MOdeling for Lifelong Language Learningcited by 156
  23. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  24. 2020PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Trackingcited by 104
  25. 2021Sparks: Inspiration for Science Writing using Language Modelscited by 80
  26. 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 75
  27. 2022Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writingcited by 71
  28. 2022Creative Writing with an AI-Powered Writing Assistant: Perspectives from Professional Writerscited by 69
  29. 2017Event Representations for Automated Story Generation with Deep Neural Netscited by 62
  30. 2022Putting GPT-3's Creativity to the (Alternative Uses) Testcited by 62
  31. 2014Narrative Planning: Balancing Plot and Charactercited by 61
  32. 2017Adversarial Feature Matching for Text Generationcited by 58
  33. 2020RecipeGPT: Generative Pre-training Based Cooking Recipe Generation and Evaluation Systemcited by 34
  34. 2017A Hybrid Convolutional Variational Autoencoder for Text Generationcited by 33
  35. 2022The Ethical Need for Watermarks in Machine-Generated Languagecited by 30
  36. 2021Copyright in Generative Deep Learningcited by 29
  37. 2021Creativity and Machine Learning: A Surveycited by 21
  38. 2021Multiversal views on language modelscited by 6

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 18; 2025: 21; 2026: 9.

57 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  3. 2023A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writingcited by 98
  4. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 96
  5. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 85
  6. 2025Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoningcited by 56
  7. 2025AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decaycited by 36
  8. 2024Assessing and Understanding Creativity in Large Language Modelscited by 35
  9. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Userscited by 32
  10. 2024Creativity Has Left the Chat: The Price of Debiasing Language Modelscited by 29
  11. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 28
  12. 2023Citation: A Key to Building Responsible and Accountable Large Language Modelscited by 20
38 more
  1. 2024MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflowscited by 19
  2. 2023BRAINTEASER: Lateral Thinking Puzzles for Large Language Modelscited by 18
  3. 2024Can AI Be as Creative as Humans?cited by 17
  4. 2025Let's Verify Math Questions Step by Stepcited by 17
  5. 2024Player-Driven Emergence in LLM-Driven Game Narrativecited by 16
  6. 2024Enhancing Creativity in Large Language Models through Associative Thinking Strategiescited by 13
  7. 2023Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challengescited by 10
  8. 2024Assessing the Creativity of LLMs in Proposing Novel Solutions to Mathematical Problemscited by 9
  9. 2026StoryScope: Investigating idiosyncrasies in AI fictioncited by 9
  10. 2024SARD: A Human-AI Collaborative Story Generationcited by 8
  11. 2024Towards Building Specialized Generalist AI with System 1 and System 2 Fusioncited by 8
  12. 2025IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignmentcited by 8
  13. 2025Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Modelscited by 7
  14. 2024Creative Problem Solving in Large Language and Vision Models -- What Would it Take?cited by 6
  15. 2025Probing and Inducing Combinational Creativity in Vision-Language Modelscited by 6
  16. 2025POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Modelscited by 6
  17. 2026Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuningcited by 6
  18. 2025POPI: Personalizing LLMs via Optimized Natural Language Preference Inferencecited by 5
  19. 2023AI Risk Profiles: A Standards Proposal for Pre-Deployment AI Risk Disclosurescited by 4
  20. 2024Training Foundation Models as Data Compression: On Information, Model Weights and Copyright Lawcited by 4
  21. 2025Are Large Language Models Good Temporal Graph Learners?cited by 3
  22. 2024Towards a Fully Interpretable and More Scalable RSA Model for Metaphor Understandingcited by 2
  23. 2024The creative psychometric item generator: a framework for item generation and validation using large language modelscited by 2
  24. 2024Thanos: Enhancing Conversational Agents with Skill-of-Mind-Infused Large Language Modelcited by 2
  25. 2025DiffSampling: Enhancing Diversity and Accuracy in Neural Text Generationcited by 2
  26. 2025Code-driven Number Sequence Calculation: Enhancing the inductive Reasoning Abilities of Large Language Modelscited by 2
  27. 2025Multi-Stakeholder Alignment in LLM-Powered Collaborative AI Systems: A Multi-Agent Framework for Intelligent Tutoringcited by 2
  28. 2026When AI Persuades: Adversarial Explanation Attacks on Human Trust in AI-Assisted Decision Makingcited by 2
  29. 2023Advancing Perception in Artificial Intelligence through Principles of Cognitive Sciencecited by 1
  30. 2024Chaos-based reinforcement learning with TD3cited by 1
  31. 2025Thinking Outside the (Gray) Box: A Context-Based Score for Assessing Value and Originality in Neural Text Generationcited by 1
  32. 2025ConspEmoLLM-v2: A robust and stable model to detect sentiment-transformed conspiracy theoriescited by 1
  33. 2025A Comparative Approach to Assessing Linguistic Creativity of Large Language Models and Humanscited by 1
  34. 2025Conversational Agents for Building Energy Efficiency -- Advising Housing Cooperatives in Stockholm on Reducing Energy Consumptioncited by 1
  35. 2026Shared Nature, Unique Nurture: PRISM for Pluralistic Reasoning via In-context Structure Modelingcited by 1
  36. 2026On the Creativity of AI Agentscited by 1
  37. 2024TEL'M: Test and Evaluation of Language Modelscited by 0
  38. 2024Initial Development and Evaluation of the Creative Artificial Intelligence through Recurring Developments and Determinations (CAIRDD) Systemcited by 0

50 most cited of 57 All 57 (JSON)