# Generate and Revise: Reinforcement Learning in Neural Poetry

arXiv: https://arxiv.org/abs/2102.04114 · HTML: https://mlsanity.com/paper/2102.04114/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [A Deep Reinforced Model for Abstractive Summarization](https://mlsanity.com/paper/1705.04304/) (2017)
- [Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting](https://mlsanity.com/paper/1805.11080/) (2018)
- [Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation](https://mlsanity.com/paper/1805.08298/) (2018)
- [Ranking Sentences for Extractive Summarization with Reinforcement Learning](https://mlsanity.com/paper/1802.08636/) (2018)
- [Do Massively Pretrained Language Models Make Better Storytellers?](https://mlsanity.com/paper/1909.10705/) (2019)
- [Generating Sentences by Editing Prototypes](https://mlsanity.com/paper/1709.08878/) (2017)
- [Retrieve and Refine: Improved Sequence Generation Models For Dialogue](https://mlsanity.com/paper/1808.04776/) (2018)
- [Deep-speare: A Joint Neural Model of Poetic Language, Meter and Rhyme](https://mlsanity.com/paper/1807.03491/) (2018)
- [Generating Chinese Classical Poems with RNN Encoder-Decoder](https://mlsanity.com/paper/1604.01537/) (2016)
- [Chinese Song Iambics Generation with Neural Attention-based Model](https://mlsanity.com/paper/1604.06274/) (2016)
- [Proximal Policy Optimization and its Dynamic Version for Sequence Generation](https://mlsanity.com/paper/1808.07982/) (2018)
- [Machine Translation for Machines: the Sentiment Classification Use Case](https://mlsanity.com/paper/1910.00478/) (2019)
- [Pseudo Random Number Generation: a Reinforcement Learning approach](https://mlsanity.com/paper/1912.11531/) (2019)
- [An Unsupervised Character-Aware Neural Approach to Word and Context Representation Learning](https://mlsanity.com/paper/1908.01819/) (2019)

## Built on by (most cited first, 3 of 3)

- [Creative Data Generation: A Review Focusing on Text and Poetry](https://mlsanity.com/paper/2305.08493/) (2023)
- [REFFLY: Melody-Constrained Lyrics Editing Model](https://mlsanity.com/paper/2409.00292/) (2024)
- [Romantic-Computing](https://mlsanity.com/paper/2206.11864/) (2022)

## Abstract (arXiv)

Writers, poets, singers usually do not create their compositions in just one breath. Text is revisited, adjusted, modified, rephrased, even multiple times, in order to better convey meanings, emotions and feelings that the author wants to express. Amongst the noble written arts, Poetry is probably the one that needs to be elaborated the most, since the composition has to formally respect predefined meter and rhyming schemes. In this paper, we propose a framework to generate poems that are repeatedly revisited and corrected, as humans do, in order to improve their overall quality. We frame the problem of revising poems in the context of Reinforcement Learning and, in particular, using Proximal Policy Optimization. Our model generates poems from scratch and it learns to progressively adjust the generated text in order to match a target criterion. We evaluate this approach in the case of matching a rhyming scheme, without having any information on which words are responsible of creating rhymes and on how to coherently alter the poem words. The proposed framework is general and, with an appropriate reward shaping, it can be applied to other text generation problems.