# Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories

arXiv: https://arxiv.org/abs/2605.26492 · HTML: https://mlsanity.com/paper/2605.26492/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 26 May 2026.

## Builds on (most cited first, 9 of 9)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023)
- [Generative artificial intelligence enhances creativity but reduces the diversity of novel content](https://mlsanity.com/paper/2312.00506/) (2023)
- [A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'](https://mlsanity.com/paper/2410.12954/) (2024)
- [NoveltyBench: Evaluating Language Models for Humanlike Diversity](https://mlsanity.com/paper/2504.05228/) (2025)
- [Modifying Large Language Model Post-Training for Diverse Creative Writing](https://mlsanity.com/paper/2503.17126/) (2025)
- [Detecting Mode Collapse in Language Models via Narration](https://mlsanity.com/paper/2402.04477/) (2024)
- [Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs](https://mlsanity.com/paper/2510.01218/) (2025)
- [The GPT-WritingPrompts Dataset: A Comparative Analysis of Character Portrayal in Short Stories](https://mlsanity.com/paper/2406.16767/) (2024)

## Abstract (arXiv)

LLM-generated stories are a popular use case, but they show very low variability. We sample 20,000 total stories from four current models using five prompts. We find that 11 words occur in 88.3% of generated stories, with little difference between models. These words include names (Elias, Mara, Elara), settings (lighthouses), and professions (clockmaker, librarian). These tokens do not often occur in published literature nor pre-training data, but they are found in preference data that is likely to have been used by all current models. Surprisingly, these "lighthouse" stories are infrequent when compared with the average post-training story, much of which contains references to copyrighted characters or adult content. This result demonstrates the potentially disproportionate impact of small datasets combined with powerful alignment algorithms.