# ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn Language Models

arXiv: https://arxiv.org/abs/2510.14077 · HTML: https://mlsanity.com/paper/2510.14077/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 15 Oct 2025.

## Builds on (most cited first, 13 of 13)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020)
- [Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs](https://mlsanity.com/paper/2406.15927/) (2024)
- [Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis](https://mlsanity.com/paper/2210.04714/) (2022)
- [LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History](https://mlsanity.com/paper/2402.18216/) (2024)
- [SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking](https://mlsanity.com/paper/2306.05426/) (2023)

## Built on by (most cited first, 6 of 6)

- [Lost in Stories: Consistency Bugs in Long Story Generation by LLMs](https://mlsanity.com/paper/2603.05890/) (2026)
- [Do LLMs Benefit From Their Own Words?](https://mlsanity.com/paper/2602.24287/) (2026)
- [Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models](https://mlsanity.com/paper/2605.30251/) (2026)
- [Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap](https://mlsanity.com/paper/2605.24432/) (2026)
- [What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction](https://mlsanity.com/paper/2609.05882/) (2026)
- [Evaluating Language Model Safety Across Long Adversarial Conversations](https://mlsanity.com/paper/2609.38357/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) suffer significant performance degradation in multi-turn conversations when information is presented incrementally. Given that multi-turn conversations characterize everyday interactions with LLMs, this degradation poses a severe challenge to real world usability. We hypothesize that abrupt increases in model uncertainty signal misalignment in multi-turn LLM interactions, and we exploit this insight to dynamically realign conversational context. We introduce ERGO (Entropy-guided Resetting for Generation Optimization), which continuously quantifies internal uncertainty via Shannon entropy over next token distributions and triggers adaptive prompt consolidation when a sharp spike in entropy is detected. By treating uncertainty as a first class signal rather than a nuisance to eliminate, ERGO embraces variability in language and modeling, representing and responding to uncertainty. In multi-turn tasks with incrementally revealed instructions, ERGO yields a 56.6% average performance gain over standard baselines, increases aptitude (peak performance capability) by 24.7%, and decreases unreliability (variability in performance) by 35.3%, demonstrating that uncertainty aware interventions can improve both accuracy and reliability in conversational AI.