# SAGE: A Hierarchical Framework for Evaluating Interpretive Literary Quality in Narratives

arXiv: https://arxiv.org/abs/2609.06611 · HTML: https://mlsanity.com/paper/2609.06611/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 6 Sep 2026.

## Builds on (most cited first, 11 of 11)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](https://mlsanity.com/paper/1905.02947/) (2019)
- [Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation](https://mlsanity.com/paper/2208.11646/) (2022)
- [The emotional arcs of stories are dominated by six basic shapes](https://mlsanity.com/paper/1606.07772/) (2016)
- [SCORE: Story Coherence and Retrieval Enhancement for AI Narratives](https://mlsanity.com/paper/2503.23512/) (2025)
- [LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigm](https://mlsanity.com/paper/2502.19103/) (2025)
- [NarraBench: A Comprehensive Framework for Narrative Benchmarking](https://mlsanity.com/paper/2510.09869/) (2025)

## Abstract (arXiv)

Assessing the literary quality of narratives requires evaluating interpretive dimensions (cultural representation, emotional depth, and philosophical engagement) that existing NLG metrics cannot measure. We introduce SAGE, a six-layer evaluation framework that separates rule-based assessment of observable textual properties from LLM-based evaluation of interpretive qualities drawn from cultural theory, affect theory, and existentialist philosophy. Each interpretive layer is assessed through multi-round iterative LLM evaluation with independent cross-validation, achieving measurement-grade reliability (98.8% convergence, >94% inter-rater agreement) stable across evaluator models. Validated on 600 evaluations across 100 short stories, our central finding is a systematic capability boundary: emotional-psychological representation approaches human levels, while cultural critique and philosophical depth exhibit approximately double the gap. LLM-generated narratives score below even commercial genre fiction on all three layers. We interpret this as a boundary between pattern-reproducible literary capacities learnable from training corpora and stance-requiring ones demanding cultural positioning and philosophical engagement that pattern matching alone cannot provide.