# Fabula Entropy Indexing: Objective Measures of Story Coherence

arXiv: https://arxiv.org/abs/2104.07472 · HTML: https://mlsanity.com/paper/2104.07472/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 43 references that are in the map.
- First posted on arXiv 23 Mar 2021.

## Builds on (most cited first, 9 of 9)

- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking](https://mlsanity.com/paper/2004.14967/) (2020)
- [Event Representations for Automated Story Generation with Deep Neural Nets](https://mlsanity.com/paper/1706.01331/) (2017)
- [Narrative Planning: Balancing Plot and Character](https://mlsanity.com/paper/1401.3841/) (2014)
- [STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation](https://mlsanity.com/paper/2010.01717/) (2020)
- [Controllable Neural Story Plot Generation via Reward Shaping](https://mlsanity.com/paper/1809.10736/) (2018)
- [Modeling Protagonist Emotions for Emotion-Aware Storytelling](https://mlsanity.com/paper/2010.06822/) (2020)
- [Automated Storytelling via Causal, Commonsense Plot Ordering](https://mlsanity.com/paper/2009.00829/) (2020)
- [DeepTingle](https://mlsanity.com/paper/1705.03557/) (2017)

## Built on by (most cited first, 7 of 7)

- [Open-world Story Generation with Structured Knowledge Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2212.04634/) (2022)
- [Where Do People Tell Stories Online? Story Detection Across Online Communities](https://mlsanity.com/paper/2311.09675/) (2023)
- [Cut the CARP: Fishing for zero-shot story evaluation](https://mlsanity.com/paper/2110.03111/) (2021)
- [TropeTwist: Trope-based Narrative Structure Generation](https://mlsanity.com/paper/2204.09672/) (2022)
- [Automated Story Generation as Question-Answering](https://mlsanity.com/paper/2112.03808/) (2021)
- [Re:Verse -- Can Your VLM Read a Manga?](https://mlsanity.com/paper/2508.08508/) (2025)
- [Rushes: A Human Preference Dataset for Pluralistic Alignment](https://mlsanity.com/paper/2607.20767/) (2026)

## Abstract (arXiv)

Automated story generation remains a difficult area of research because it lacks strong objective measures. Generated stories may be linguistically sound, but in many cases suffer poor narrative coherence required for a compelling, logically-sound story. To address this, we present Fabula Entropy Indexing (FEI), an evaluation method to assess story coherence by measuring the degree to which human participants agree with each other when answering true/false questions about stories. We devise two theoretically grounded measures of reader question-answering entropy, the entropy of world coherence (EWC), and the entropy of transitional coherence (ETC), focusing on global and local coherence, respectively. We evaluate these metrics by testing them on human-written stories and comparing against the same stories that have been corrupted to introduce incoherencies. We show that in these controlled studies, our entropy indices provide a reliable objective measure of story coherence.