# PoeTone: A Framework for Constrained Generation of Structured Chinese Songci with LLMs

arXiv: https://arxiv.org/abs/2508.02515 · HTML: https://mlsanity.com/paper/2508.02515/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Chinese Poetry Generation with Planning based Neural Network](https://mlsanity.com/paper/1610.09889/) (2016)
- [CharPoet: A Chinese Classical Poetry Generation System Based on Token-free LLM](https://mlsanity.com/paper/2401.03512/) (2024)
- [Generating Chinese Classical Poems with RNN Encoder-Decoder](https://mlsanity.com/paper/1604.01537/) (2016)
- [GPT-based Generation for Classical Chinese Poetry](https://mlsanity.com/paper/1907.00151/) (2019)
- [Generating Major Types of Chinese Classical Poetry in a Uniformed Framework](https://mlsanity.com/paper/2003.11528/) (2020)
- [Chinese Poetry Generation with a Working Memory Model](https://mlsanity.com/paper/1809.04306/) (2018)
- [Chinese Poetry Generation with a Salient-Clue Mechanism](https://mlsanity.com/paper/1809.04313/) (2018)

## Built on by (most cited first, 3 of 3)

- [MetricalARGS: A Taxonomy for Studying Metrical Poetry with LLMs](https://mlsanity.com/paper/2510.08188/) (2025)
- [Translating Classical Poetry into Modern Prose](https://mlsanity.com/paper/2606.02806/) (2026)
- [Neo-Classic: A Benchmark for Evaluating Linguistic-Aesthetic Reasoning in Classical Chinese Poetry](https://mlsanity.com/paper/2609.19154/) (2026)

## Abstract (arXiv)

This paper presents a systematic investigation into the constrained generation capabilities of large language models (LLMs) in producing Songci, a classical Chinese poetry form characterized by strict structural, tonal, and rhyme constraints defined by Cipai templates. We first develop a comprehensive, multi-faceted evaluation framework that includes: (i) a formal conformity score, (ii) automated quality assessment using LLMs, (iii) human evaluation, and (iv) classification-based probing tasks. Using this framework, we evaluate the generative performance of 18 LLMs, including 3 proprietary models and 15 open-source models across 4 families, under five prompting strategies: zero-shot, one-shot, completion-based, instruction-based, and chain-of-thought. Finally, we propose a Generate-Critic architecture in which the evaluation framework functions as an automated critic. Leveraging the critic's feedback as a scoring function for best-of-N selection, we fine-tune 3 lightweight open-source LLMs via supervised fine-tuning (SFT), resulting in improvements of up to 5.88% in formal conformity. Our findings offer new insights into the generative strengths and limitations of LLMs in producing culturally significant and formally constrained literary texts.