# Understanding Literary Texts by LLMs: A Case Study of Ancient Chinese Poetry

arXiv: https://arxiv.org/abs/2409.00060 · HTML: https://mlsanity.com/paper/2409.00060/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 22 Aug 2024.

## Builds on (most cited first, 10 of 10)

- [Re3: Generating Longer Stories With Recursive Reprompting and Revision](https://mlsanity.com/paper/2210.06774/) (2022)
- [Demystifying Neural Style Transfer](https://mlsanity.com/paper/1701.01036/) (2017)
- [Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing](https://mlsanity.com/paper/2210.13669/) (2022)
- [MixPoet: Diverse Poetry Generation via Learning Controllable Mixed Latent Space](https://mlsanity.com/paper/2003.06094/) (2020)
- [GROVE: A Retrieval-augmented Complex Story Generation Framework with A Forest of Evidence](https://mlsanity.com/paper/2310.05388/) (2023)
- [Towards Inter-character Relationship-driven Story Generation](https://mlsanity.com/paper/2211.00676/) (2022)
- [Evaluating LLMs for Quotation Attribution in Literary Texts: A Case Study of LLaMa3](https://mlsanity.com/paper/2406.11380/) (2024)
- [Witscript: A System for Generating Improvised Jokes in a Conversation](https://mlsanity.com/paper/2302.02008/) (2023)
- [Witscript 2: A System for Generating Improvised Jokes Without Wordplay](https://mlsanity.com/paper/2302.03036/) (2023)
- [How to Evaluate Coreference in Literary Texts?](https://mlsanity.com/paper/2401.00238/) (2023)

## Built on by (most cited first, 4 of 4)

- [Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs](https://mlsanity.com/paper/2505.18152/) (2025)
- [HistoLens: An LLM-Powered Framework for Multi-Layered Analysis of Historical Texts -- A Case Application of Yantie Lun](https://mlsanity.com/paper/2411.09978/) (2024)
- [Analyzing Nobel Prize Literature with Large Language Models](https://mlsanity.com/paper/2410.18142/) (2024)
- [Do Large Language Models Perform Well on Comprehending Poetic Logic in Modern Chinese Poetry?](https://mlsanity.com/paper/2608.21827/) (2026)

## Abstract (arXiv)

The birth and rapid development of large language models (LLMs) have caused quite a stir in the field of literature. Once considered unattainable, AI's role in literary creation is increasingly becoming a reality. In genres such as poetry, jokes, and short stories, numerous AI tools have emerged, offering refreshing new perspectives. However, it's difficult to further improve the quality of these works. This is primarily because understanding and appreciating a good literary work involves a considerable threshold, such as knowledge of literary theory, aesthetic sensibility, interdisciplinary knowledge. Therefore, authoritative data in this area is quite lacking. Additionally, evaluating literary works is often complex and hard to fully quantify, which directly hinders the further development of AI creation. To address this issue, this paper attempts to explore the mysteries of literary texts from the perspective of LLMs, using ancient Chinese poetry as an example for experimentation. First, we collected a variety of ancient poems from different sources and had experts annotate a small portion of them. Then, we designed a range of comprehension metrics based on LLMs to evaluate all these poems. Finally, we analyzed the correlations and differences between various poem collections to identify literary patterns. Through our experiments, we observed a series of enlightening phenomena that provide technical support for the future development of high-level literary creation based on LLMs.