SAGE: A Hierarchical Framework for Evaluating Interpretive Literary Quality in Narratives
Not on the map. Why?
Abstract (arXiv)
Assessing the literary quality of narratives requires evaluating interpretive dimensions (cultural representation, emotional depth, and philosophical engagement) that existing NLG metrics cannot measure. We introduce SAGE, a six-layer evaluation framework that separates rule-based assessment of observable textual properties from LLM-based evaluation of interpretive qualities drawn from cultural theory, affect theory, and existentialist philosophy. Each interpretive layer is assessed through multi-round iterative LLM evaluation with independent cross-validation, achieving measurement-grade reliability (98.8% convergence, >94% inter-rater agreement) stable across evaluator models. Validated on 600 evaluations across 100 short stories, our central finding is a systematic capability boundary: emotional-psychological representation approaches human levels, while cultural critique and philosophical depth exhibit approximately double the gap. LLM-generated narratives score below even commercial genre fiction on all three layers. We interpret this as a boundary between pattern-reproducible literary capacities learnable from training corpora and stance-requiring ones demanding cultural positioning and philosophical engagement that pattern matching alone cannot provide.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 35 references that are in the map. First posted on arXiv 6 Sep 2026.
Builds on
11 of 35 references.
References · 11
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
- 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
- 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
- 2017Why We Need New Evaluation Metrics for NLGcited by 233
- 2019Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advancescited by 119
- 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 75
- 2016The emotional arcs of stories are dominated by six basic shapescited by 71
- 2025SCORE: Story Coherence and Retrieval Enhancement for AI Narrativescited by 42
- 2025LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigmcited by 14
- 2025NarraBench: A Comprehensive Framework for Narrative Benchmarkingcited by 13
Built on by
Not cited yet.