# Large Language Models for Classical Chinese Poetry Translation: Benchmarking, Evaluating, and Improving

arXiv: https://arxiv.org/abs/2408.09945 · HTML: https://mlsanity.com/paper/2408.09945/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2024.

## Built on by (most cited first, 7 of 7)

- [Survey of Cultural Awareness in Language Models: Text and Beyond](https://mlsanity.com/paper/2411.00860/) (2024)
- [Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering](https://mlsanity.com/paper/2410.01401/) (2024)
- [HistoLens: An LLM-Powered Framework for Multi-Layered Analysis of Historical Texts -- A Case Application of Yantie Lun](https://mlsanity.com/paper/2411.09978/) (2024)
- [Make Imagination Clearer! Stable Diffusion-based Visual Imagination for Multimodal Machine Translation](https://mlsanity.com/paper/2412.12627/) (2024)
- [SlangDIT: Benchmarking LLMs in Interpretative Slang Translation](https://mlsanity.com/paper/2505.14181/) (2025)
- [System Report for CCL25-Eval Task 5: New Dataset and LoRA-Fine-Tuned Qwen2.5](https://mlsanity.com/paper/2606.12392/) (2026)
- [Automated evaluation of LLMs for effective machine translation of Mandarin Chinese to English](https://mlsanity.com/paper/2603.09998/) (2026)

## Abstract (arXiv)

Different from the traditional translation tasks, classical Chinese poetry translation requires both adequacy and fluency in translating culturally and historically significant content and linguistic poetic elegance. Large language models (LLMs) with impressive multilingual capabilities may bring a ray of hope to achieve this extreme translation demand. This paper first introduces a suitable benchmark (PoetMT) where each Chinese poetry has a recognized elegant translation. Meanwhile, we propose a new metric based on GPT-4 to evaluate the extent to which current LLMs can meet these demands. Our empirical evaluation reveals that the existing LLMs fall short in the challenging task. Hence, we propose a Retrieval-Augmented Machine Translation (RAT) method which incorporates knowledge related to classical poetry for advancing the translation of Chinese Poetry in LLMs. Experimental results show that RAT consistently outperforms all comparison methods regarding wildly used BLEU, COMET, BLEURT, our proposed metric, and human evaluation.