# Avoidance Decoding for Diverse Multi-Branch Story Generation

arXiv: https://arxiv.org/abs/2509.02170 · HTML: https://mlsanity.com/paper/2509.02170/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Builds on (most cited first, 24 of 24)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search](https://mlsanity.com/paper/1704.07138/) (2017)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs](https://mlsanity.com/paper/2407.01082/) (2024)
- [NoveltyBench: Evaluating Language Models for Humanlike Diversity](https://mlsanity.com/paper/2504.05228/) (2025)
- [We're Different, We're the Same: Creative Homogeneity Across LLMs](https://mlsanity.com/paper/2501.19361/) (2025)
- [Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation](https://mlsanity.com/paper/2407.18698/) (2024)
- [Collective Critics for Creative Story Generation](https://mlsanity.com/paper/2410.02428/) (2024)
- [Curiosity-Driven Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2501.11463/) (2025)
- [Goal-Directed Story Generation: Augmenting Generative Language Models with Reinforcement Learning](https://mlsanity.com/paper/2112.08593/) (2021)
- [GROVE: A Retrieval-augmented Complex Story Generation Framework with A Forest of Evidence](https://mlsanity.com/paper/2310.05388/) (2023)
- [Drift: Decoding-time Personalized Alignments with Implicit User Preferences](https://mlsanity.com/paper/2502.14289/) (2025)
- [LongStory: Coherent, Complete and Length Controlled Long story Generation](https://mlsanity.com/paper/2311.15208/) (2023)
- [A Character-Centric Creative Story Generation via Imagination](https://mlsanity.com/paper/2409.16667/) (2024)
- [Is Depth All You Need? An Exploration of Iterative Reasoning in LLMs](https://mlsanity.com/paper/2502.10858/) (2025)
- [Multi-Novelty: Improve the Diversity and Novelty of Contents Generated by Large Language Models via inference-time Multi-Views Brainstorming](https://mlsanity.com/paper/2502.12700/) (2025)
- [COS(M+O)S: Curiosity and RL-Enhanced MCTS for Exploring Story Space via Language Models](https://mlsanity.com/paper/2501.17104/) (2025)

## Built on by (most cited first, 6 of 6)

- [Where You Inject Diversity Matters: A Unified Framework for Diverse Generation](https://mlsanity.com/paper/2606.10302/) (2026)
- [A Universal Avoidance Method for Diverse Multi-branch Generation](https://mlsanity.com/paper/2604.17323/) (2026)
- [Do Large Language Models Always Tell The Same Stories?](https://mlsanity.com/paper/2606.17350/) (2026)
- [Telling Speculative Stories to Help Humans Imagine the Harms of Healthcare AI](https://mlsanity.com/paper/2510.14718/) (2025)
- [Are You Sure You're Sure? On the Impact of Instruction Tuning on Confidence and Lexical Diversity](https://mlsanity.com/paper/2608.13430/) (2026)
- [On Epistemic Diversity in Large Language Models](https://mlsanity.com/paper/2609.04835/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) often generate repetitive and monotonous outputs, especially in tasks like story generation, due to limited creative diversity when given the same input prompt. To address this challenge, we propose a novel decoding strategy, Avoidance Decoding, that modifies token logits by penalizing similarity to previously generated outputs, thereby encouraging more diverse multi-branch stories. This penalty adaptively balances two similarity measures: (1) Concept-level Similarity Penalty, which is prioritized in early stages to diversify initial story concepts, and (2) Narrative-level Similarity Penalty, which is increasingly emphasized later to ensure natural yet diverse plot development. Notably, our method achieves up to 2.6 times higher output diversity and reduces repetition by an average of 30% compared to strong baselines, while effectively mitigating text degeneration. Furthermore, we reveal that our method activates a broader range of neurons, demonstrating that it leverages the model's intrinsic creativity.