# TPPoet: Transformer-Based Persian Poem Generation using Minimal Data and Advanced Decoding Techniques

arXiv: https://arxiv.org/abs/2312.02125 · HTML: https://mlsanity.com/paper/2312.02125/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 4 Dec 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Textbooks Are All You Need](https://mlsanity.com/paper/2306.11644/) (2023)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [TinyStories: How Small Can Language Models Be and Still Speak Coherent English?](https://mlsanity.com/paper/2305.07759/) (2023)
- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing](https://mlsanity.com/paper/2006.03236/) (2020)
- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [Language GANs Falling Short](https://mlsanity.com/paper/1811.02549/) (2018)
- [Jointly Measuring Diversity and Quality in Text Generation Models](https://mlsanity.com/paper/1904.03971/) (2019)
- [Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder](https://mlsanity.com/paper/2304.04052/) (2023)
- [DTrOCR: Decoder-only Transformer for Optical Character Recognition](https://mlsanity.com/paper/2308.15996/) (2023)
- [ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models](https://mlsanity.com/paper/2212.10474/) (2022)
- [Generating Chinese Classical Poems with RNN Encoder-Decoder](https://mlsanity.com/paper/1604.01537/) (2016)
- [A Frustratingly Simple Decoding Method for Neural Text Generation](https://mlsanity.com/paper/2305.12675/) (2023)
- [GAN Computers Generate Arts? A Survey on Visual Arts, Music, and Literary Text Generation using Generative Adversarial Network](https://mlsanity.com/paper/2108.03857/) (2021)
- [Introducing Aspects of Creativity in Automatic Poetry Generation](https://mlsanity.com/paper/2002.02511/) (2020)
- [Generating Thematic Chinese Poetry using Conditional Variational Autoencoders with Hybrid Decoders](https://mlsanity.com/paper/1711.07632/) (2017)
- [Prose2Poem: The Blessing of Transformers in Translating Prose to Persian Poetry](https://mlsanity.com/paper/2109.14934/) (2021)
- [SP-GPT2: Semantics Improvement in Vietnamese Poetry Generation](https://mlsanity.com/paper/2110.15723/) (2021)
- [Can Machine Generate Traditional Chinese Poetry? A Feigenbaum Test](https://mlsanity.com/paper/1606.05829/) (2016)
- [Modern French Poetry Generation with RoBERTa and GPT-2](https://mlsanity.com/paper/2212.02911/) (2022)

## Built on by (most cited first, 2 of 2)

- [Bhaasha, Bhasa, Zaban: A Survey for Low-Resourced Languages in South Asia -- Current Stage and Challenges](https://mlsanity.com/paper/2509.11570/) (2025)
- [Echoes Across Centuries: Phonetic Signatures of Persian Poets](https://mlsanity.com/paper/2603.14443/) (2026)

## Abstract (arXiv)

Recent advances in language models (LMs), have demonstrated significant efficacy in tasks related to the arts and humanities. While LMs have exhibited exceptional performance across a wide range of natural language processing tasks, there are notable challenges associated with their utilization on small datasets and their ability to replicate more creative human capacities. In this study, we aim to address these challenges by training a Persian classical poetry generation model using a transformer architecture on a specialized dataset with no pretraining. Additionally, we propose a novel decoding method to enhance coherence and meaningfulness in the generated poetry, effectively managing the tradeoff between diversity and quality. Furthermore, the results of our training approach and the proposed decoding method are evaluated through comprehensive set of automatic and human evaluations and showed its superior capability to generate coherent and meaningful poetry in compare to other decoding methods and an existing Persian large language model (LLM).