# Clarify, Abstain or Answer? Strategising in Conversation with Belief-Augmented Generation

arXiv: https://arxiv.org/abs/2605.25831 · HTML: https://mlsanity.com/paper/2605.25831/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 70 references that are in the map.
- First posted on arXiv 25 May 2026.

## Builds on (most cited first, 30 of 36)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023)
- [ASQA: Factoid Questions Meet Long-Form Answers](https://mlsanity.com/paper/2204.06092/) (2022)
- [Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering](https://mlsanity.com/paper/2307.16877/) (2023)
- [SituatedQA: Incorporating Extra-Linguistic Contexts into QA](https://mlsanity.com/paper/2109.06157/) (2021)
- [Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation](https://mlsanity.com/paper/2005.10283/) (2020)
- [Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration](https://mlsanity.com/paper/2305.13626/) (2023)
- [Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling](https://mlsanity.com/paper/2311.08718/) (2023)
- [SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales](https://mlsanity.com/paper/2405.20974/) (2024)
- [STaR-GATE: Teaching Language Models to Ask Clarifying Questions](https://mlsanity.com/paper/2403.19154/) (2024)
- [Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs](https://mlsanity.com/paper/2311.09469/) (2023)
- [Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?](https://mlsanity.com/paper/2405.16908/) (2024)
- [Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions](https://mlsanity.com/paper/2410.13788/) (2024)
- [Uncertainty in Natural Language Generation: From Theory to Applications](https://mlsanity.com/paper/2307.15703/) (2023)
- [Aligning Language Models to Explicitly Handle Ambiguity](https://mlsanity.com/paper/2404.11972/) (2024)
- [Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2211.07634/) (2022)
- [What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability](https://mlsanity.com/paper/2305.11707/) (2023)
- [It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk](https://mlsanity.com/paper/2310.01387/) (2023)
- [Grounding Gaps in Language Model Generations](https://mlsanity.com/paper/2311.09144/) (2023)
- [Joint Passage Ranking for Diverse Multi-Answer Retrieval](https://mlsanity.com/paper/2104.08445/) (2021)
- [Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions](https://mlsanity.com/paper/2402.06509/) (2024)
- [Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction](https://mlsanity.com/paper/2011.13137/) (2020)
- [Better Instruction-Following Through Minimum Bayes Risk](https://mlsanity.com/paper/2410.02902/) (2024)
- [Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?](https://mlsanity.com/paper/2402.16102/) (2024)
- [CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering](https://mlsanity.com/paper/2502.01523/) (2025)
- [Conversation as Action Under Uncertainty](https://mlsanity.com/paper/1301.3883/) (2013)
- [To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity](https://mlsanity.com/paper/2407.17125/) (2024)
- [RAcQUEt: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMs](https://mlsanity.com/paper/2412.13835/) (2024)

## Built on by (most cited first, 1 of 1)

- [What Is a Clarification? Estimating Model-Dependent Input Ambiguity](https://mlsanity.com/paper/2610.20052/) (2026)

## Abstract (arXiv)

Large language models (LLMs) define a distribution over text, which can be viewed as a probabilistic representation of uncertainty: sampling K responses yields a belief state - responses a model deems plausible. Existing work exploits this representation for narrow tasks like either decoding or selective prediction, and often requires manual interventions, not controlling generation directly. We propose Belief-Augmented Generation (BAG): grounding LLMs in their own belief state via the prompt and letting them reason over these K samples to decide on and execute a conversational strategy: clarify, abstain, or answer. In a multi-turn ambiguous question answering (QA) setting, we find that LLMs by default rarely clarify or abstain, ignoring uncertainty about the input (aleatoric) or facts (epistemic). BAG improves QA accuracy across six models and yields strategy decisions more faithful to their belief state than prompt-only baselines. Disentangling when to clarify from when to abstain, however, remains challenging.