# SP-GPT2: Semantics Improvement in Vietnamese Poetry Generation

arXiv: https://arxiv.org/abs/2110.15723 · HTML: https://mlsanity.com/paper/2110.15723/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 10 Oct 2021.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Generating Chinese Classical Poems with RNN Encoder-Decoder](https://mlsanity.com/paper/1604.01537/) (2016)
- [Generating Thematic Chinese Poetry using Conditional Variational Autoencoders with Hybrid Decoders](https://mlsanity.com/paper/1711.07632/) (2017)
- [Chinese Song Iambics Generation with Neural Attention-based Model](https://mlsanity.com/paper/1604.06274/) (2016)
- [Can Machine Generate Traditional Chinese Poetry? A Feigenbaum Test](https://mlsanity.com/paper/1606.05829/) (2016)

## Built on by (most cited first, 5 of 5)

- [Improving Transformers with Probabilistic Attention Keys](https://mlsanity.com/paper/2110.08678/) (2021)
- [SikuGPT: A Generative Pre-trained Model for Intelligent Information Processing of Ancient Texts from the Perspective of Digital Humanities](https://mlsanity.com/paper/2304.07778/) (2023)
- [TPPoet: Transformer-Based Persian Poem Generation using Minimal Data and Advanced Decoding Techniques](https://mlsanity.com/paper/2312.02125/) (2023)
- [Vietnamese Poem Generation & The Prospect Of Cross-Language Poem-To-Poem Translation](https://mlsanity.com/paper/2401.01078/) (2024)
- [Automated Evaluation of Meter and Rhyme in Russian Generative and Human-Authored Poetry](https://mlsanity.com/paper/2502.20931/) (2025)

## Abstract (arXiv)

Automatic text generation has garnered growing attention in recent years as an essential step towards computer creativity. Generative Pretraining Transformer 2 (GPT2) is one of the state of the art approaches that have excellent successes. In this paper, we took the first step to investigate the power of GPT2 in traditional Vietnamese poetry generation. In the earlier time, our experiment with base GPT2 was quite good at generating the poem in the proper template. Though it can learn the patterns, including rhyme and tone rules, from the training data, like almost all other text generation approaches, the poems generated still has a topic drift and semantic inconsistency. To improve the cohesion within the poems, we proposed a new model SP-GPT2 (semantic poem GPT2) which was built on the top GPT2 model and an additional loss to constrain context throughout the entire poem. For better evaluation, we examined the methods by both automatic quantitative evaluation and human evaluation. Both automatic and human evaluation demonstrated that our approach can generate poems that have better cohesion without losing the quality due to additional loss. At the same time, we are the pioneers of this topic. We released the first computational scoring module for poems generated in the template containing the style rule dictionary. Additionally, we are the first to publish a Luc-Bat dataset, including 87609 Luc Bat poems, which is equivalent to about 2.6 million sentences, combined with about 83579 poems in other styles was also published for further exploration. The code is available at https://github.com/fsoft-ailab/Poem-Generator