# Modern French Poetry Generation with RoBERTa and GPT-2

arXiv: https://arxiv.org/abs/2212.02911 · HTML: https://mlsanity.com/paper/2212.02911/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 35 references that are in the map.
- First posted on arXiv 6 Dec 2022.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [CamemBERT: a Tasty French Language Model](https://mlsanity.com/paper/1911.03894/) (2019)
- [Towards a Cleaner Document-Oriented Multilingual Crawled Corpus](https://mlsanity.com/paper/2201.06642/) (2022)
- [Deep-speare: A Joint Neural Model of Poetic Language, Meter and Rhyme](https://mlsanity.com/paper/1807.03491/) (2018)
- [Human Evaluation of Creative NLG Systems: An Interdisciplinary Survey on Recent Papers](https://mlsanity.com/paper/2108.00308/) (2021)
- [Generating Major Types of Chinese Classical Poetry in a Uniformed Framework](https://mlsanity.com/paper/2003.11528/) (2020)
- [Let's FACE it. Finnish Poetry Generation with Aesthetics and Framing](https://mlsanity.com/paper/1910.13946/) (2019)
- [The Great Misalignment Problem in Human Evaluation of NLP Methods](https://mlsanity.com/paper/2104.05361/) (2021)

## Built on by (most cited first, 1 of 1)

- [TPPoet: Transformer-Based Persian Poem Generation using Minimal Data and Advanced Decoding Techniques](https://mlsanity.com/paper/2312.02125/) (2023)

## Abstract (arXiv)

We present a novel neural model for modern poetry generation in French. The model consists of two pretrained neural models that are fine-tuned for the poem generation task. The encoder of the model is a RoBERTa based one while the decoder is based on GPT-2. This way the model can benefit from the superior natural language understanding performance of RoBERTa and the good natural language generation performance of GPT-2. Our evaluation shows that the model can create French poetry successfully. On a 5 point scale, the lowest score of 3.57 was given by human judges to typicality and emotionality of the output poetry while the best score of 3.79 was given to understandability.