# GPoeT-2: A GPT-2 Based Poem Generator

arXiv: https://arxiv.org/abs/2205.08847 · HTML: https://mlsanity.com/paper/2205.08847/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 18 May 2022.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model](https://mlsanity.com/paper/1902.04094/) (2019)
- [Leveraging Pre-trained Checkpoints for Sequence Generation Tasks](https://mlsanity.com/paper/1907.12461/) (2019)
- [Deep Learning Based Text Classification: A Comprehensive Review](https://mlsanity.com/paper/2004.03705/) (2020)
- [Learning Autocompletion from Real-World Datasets](https://mlsanity.com/paper/2011.04542/) (2020)
- [Introducing Aspects of Creativity in Automatic Poetry Generation](https://mlsanity.com/paper/2002.02511/) (2020)
- [Learning Rhyming Constraints using Structured Adversaries](https://mlsanity.com/paper/1909.06743/) (2019)

## Built on by (most cited first, 5 of 5)

- [Machine Psychology](https://mlsanity.com/paper/2303.13988/) (2023)
- [GPT Czech Poet: Generation of Czech Poetic Strophes with Language Models](https://mlsanity.com/paper/2407.12790/) (2024)
- [PoemTale Diffusion: Minimising Information Loss in Poem to Image Generation with Multi-Stage Prompt Refinement](https://mlsanity.com/paper/2507.13708/) (2025)
- [POEMetric: The Last Stanza of Humanity](https://mlsanity.com/paper/2604.03695/) (2026)
- [Raply: A profanity-mitigated rap generator](https://mlsanity.com/paper/2407.06941/) (2024)

## Abstract (arXiv)

This project aims to produce the next volume of machine-generated poetry, a complex art form that can be structured and unstructured, and carries depth in the meaning between the lines. GPoeT-2 is based on fine-tuning a state of the art natural language model (i.e. GPT-2) to generate limericks, typically humorous structured poems consisting of five lines with a AABBA rhyming scheme. With a two-stage generation system utilizing both forward and reverse language modeling, GPoeT-2 is capable of freely generating limericks in diverse topics while following the rhyming structure without any seed phrase or a posteriori constraints.Based on the automated generation process, we explore a wide variety of evaluation metrics to quantify "good poetry," including syntactical correctness, lexical diversity, and subject continuity. Finally, we present a collection of 94 categorized limericks that rank highly on the explored "good poetry" metrics to provoke human creativity.