# Do Large Language Models Perform Well on Comprehending Poetic Logic in Modern Chinese Poetry?

arXiv: https://arxiv.org/abs/2608.21827 · HTML: https://mlsanity.com/paper/2608.21827/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 43 references that are in the map.
- First posted on arXiv 22 Aug 2026.

## Builds on (most cited first, 14 of 14)

- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents](https://mlsanity.com/paper/2601.16746/) (2026)
- [SWE-QA: Can Language Models Answer Repository-level Code Questions?](https://mlsanity.com/paper/2509.14635/) (2025)
- [CharPoet: A Chinese Classical Poetry Generation System Based on Token-free LLM](https://mlsanity.com/paper/2401.03512/) (2024)
- [Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs](https://mlsanity.com/paper/2505.18152/) (2025)
- [What is the Best Way for ChatGPT to Translate Poetry?](https://mlsanity.com/paper/2406.03450/) (2024)
- [Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry](https://mlsanity.com/paper/2509.01620/) (2025)
- [Can AI Write Classical Chinese Poetry like Humans? An Empirical Study Inspired by Turing Test](https://mlsanity.com/paper/2401.04952/) (2024)
- [Understanding Literary Texts by LLMs: A Case Study of Ancient Chinese Poetry](https://mlsanity.com/paper/2409.00060/) (2024)
- [Can ChatGPT Really Understand Modern Chinese Poetry?](https://mlsanity.com/paper/2603.20851/) (2026)
- [Learning to Control Summaries with Score Ranking](https://mlsanity.com/paper/2604.17197/) (2026)
- [Calibrating Model-Based Evaluation Metrics for Summarization](https://mlsanity.com/paper/2604.17200/) (2026)
- [Exploring the Capability Boundaries of LLMs in Mastering of Chinese Chouxiang Language](https://mlsanity.com/paper/2604.15841/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have achieved significant progress across a wide range of natural language processing (NLP) tasks, yet their ability to understand literary texts, particularly modern Chinese poetry, remains largely unexplored. The unique literary characteristics of modern Chinese poetry necessitate a distinct form of reasoning for effective comprehension. Unlike conventional texts that convey clear information, the unique "poetic logic" of modern Chinese poetry requires a holistic reasoning approach that goes beyond superficial semantic analysis to be understood. However, current evaluation paradigms largely ignore this critical dimension. To address this gap, we propose Peony, the first benchmark specifically designed for evaluating the poetic logic of modern Chinese poetry. We define poetic logic as four tasks across three levels, namely stanza, line, and imagery, and systematically evaluate and analyze six mainstream LLMs based on Peony. We evaluate these models under both non-thinking and thinking configurations. The experimental results reveal the limitations of current LLMs in understanding the poetic logic of modern Chinese poetry and validate the effectiveness and necessity of Peony. Our data and code will be available.