# Does ChatGPT Have a Poetic Style?

arXiv: https://arxiv.org/abs/2410.15299 · HTML: https://mlsanity.com/paper/2410.15299/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 20 Oct 2024.

## Builds on (most cited first, 13 of 13)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Are Large Language Models Capable of Generating Human-Level Narratives?](https://mlsanity.com/paper/2407.13248/) (2024)
- [Delving into LLM-assisted writing in biomedical publications through excess vocabulary](https://mlsanity.com/paper/2406.07016/) (2024)
- [Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing](https://mlsanity.com/paper/2210.13669/) (2022)
- [Deep-speare: A Joint Neural Model of Poetic Language, Meter and Rhyme](https://mlsanity.com/paper/1807.03491/) (2018)
- [ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models](https://mlsanity.com/paper/2212.10474/) (2022)
- [Chinese Poetry Generation with Planning based Neural Network](https://mlsanity.com/paper/1610.09889/) (2016)
- [PoeLM: A Meter- and Rhyme-Controllable Language Model for Unsupervised Poetry Generation](https://mlsanity.com/paper/2205.12206/) (2022)
- [Sonnet or Not, Bot? Poetry Evaluation for Large Models and Datasets](https://mlsanity.com/paper/2406.18906/) (2024)
- [Evaluating Diversity in Automatic Poetry Generation](https://mlsanity.com/paper/2406.15267/) (2024)
- [Acrostic Poem Generation](https://mlsanity.com/paper/2010.02239/) (2020)
- [Learning Rhyming Constraints using Structured Adversaries](https://mlsanity.com/paper/1909.06743/) (2019)

## Built on by (most cited first, 12 of 12)

- [Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction](https://mlsanity.com/paper/2504.15266/) (2025)
- [Provocations from the Humanities for Generative AI Research](https://mlsanity.com/paper/2502.19190/) (2025)
- [Readers Prefer Outputs of AI Trained on Copyrighted Books over Expert Human Writers](https://mlsanity.com/paper/2510.13939/) (2025)
- [AI-generated stories favour stability over change: homogeneity and cultural stereotyping in narratives generated by gpt-4o-mini](https://mlsanity.com/paper/2507.22445/) (2025)
- [Deep Associations, High Creativity: A Simple yet Effective Metric for Evaluating Large Language Models](https://mlsanity.com/paper/2510.12110/) (2025)
- [POEMetric: The Last Stanza of Humanity](https://mlsanity.com/paper/2604.03695/) (2026)
- [AI as a Tool for Simulation-Based Experiments in Literary Studies](https://mlsanity.com/paper/2606.02293/) (2026)
- [AI Fiction in the Wild](https://mlsanity.com/paper/2606.22748/) (2026)
- [ConspEmoLLM-v2: A robust and stable model to detect sentiment-transformed conspiracy theories](https://mlsanity.com/paper/2505.14917/) (2025)
- [GhazalBench: Canonical Verse Access in LLMs across Persian Ghazals and Shakespearean Sonnets](https://mlsanity.com/paper/2603.09979/) (2026)
- [The cell as a token: high-dimensional geometry in language models and cell embeddings](https://mlsanity.com/paper/2503.20278/) (2025)
- [Training Data Size Sensitivity in Unsupervised Rhyme Recognition](https://mlsanity.com/paper/2604.08156/) (2026)

## Abstract (arXiv)

Generating poetry has become a popular application of LLMs, perhaps especially of OpenAI's widely-used chatbot ChatGPT. What kind of poet is ChatGPT? Does ChatGPT have its own poetic style? Can it successfully produce poems in different styles? To answer these questions, we prompt the GPT-3.5 and GPT-4 models to generate English-language poems in 24 different poetic forms and styles, about 40 different subjects, and in response to 3 different writing prompt templates. We then analyze the resulting 5.7k poems, comparing them to a sample of 3.7k poems from the Poetry Foundation and the Academy of American Poets. We find that the GPT models, especially GPT-4, can successfully produce poems in a range of both common and uncommon English-language forms in superficial yet noteworthy ways, such as by producing poems of appropriate lengths for sonnets (14 lines), villanelles (19 lines), and sestinas (39 lines). But the GPT models also exhibit their own distinct stylistic tendencies, both within and outside of these specific forms. Our results show that GPT poetry is much more constrained and uniform than human poetry, showing a strong penchant for rhyme, quatrains (4-line stanzas), iambic meter, first-person plural perspectives (we, us, our), and specific vocabulary like "heart," "embrace," "echo," and "whisper."