# Uncertainty-Aware Generation and Decision-Making Under Ambiguity

arXiv: https://arxiv.org/abs/2606.30578 · HTML: https://mlsanity.com/paper/2606.30578/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Builds on (most cited first, 30 of 31)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [CycleResearcher: Improving Automated Research via Automated Review](https://mlsanity.com/paper/2411.00816/) (2024)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics](https://mlsanity.com/paper/2111.09388/) (2021)
- [DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process](https://mlsanity.com/paper/2503.08569/) (2025)
- [Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation](https://mlsanity.com/paper/2108.04718/) (2021)
- [OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviews](https://mlsanity.com/paper/2412.11948/) (2024)
- [Conformal Prediction for Natural Language Processing: A Survey](https://mlsanity.com/paper/2405.01976/) (2024)
- [Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakes](https://mlsanity.com/paper/2310.10648/) (2023)
- [MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors](https://mlsanity.com/paper/2502.18940/) (2025)
- [Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors](https://mlsanity.com/paper/2407.09136/) (2024)
- [Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents](https://mlsanity.com/paper/2502.02561/) (2025)
- [Faster Minimum Bayes Risk Decoding with Confidence-based Pruning](https://mlsanity.com/paper/2311.14919/) (2023)
- [Opportunities and Challenges in Neural Dialog Tutoring](https://mlsanity.com/paper/2301.09919/) (2023)
- [ReviewAgents: Bridging the Gap Between Human and AI-Generated Paper Reviews](https://mlsanity.com/paper/2503.08506/) (2025)
- [Conformal prediction under ambiguous ground truth](https://mlsanity.com/paper/2307.09302/) (2023)
- [Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure](https://mlsanity.com/paper/2410.03781/) (2024)
- [A Diachronic Perspective on User Trust in AI under Uncertainty](https://mlsanity.com/paper/2310.13544/) (2023)
- [Mitigating Metric Bias in Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2411.03524/) (2024)
- [The Good, the Bad and the Constructive: Automatically Measuring Peer Review's Utility for Authors](https://mlsanity.com/paper/2509.04484/) (2025)
- [Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback](https://mlsanity.com/paper/2508.10795/) (2025)
- [Model-Based Minimum Bayes Risk Decoding for Text Generation](https://mlsanity.com/paper/2311.05263/) (2023)
- [Uncertainty-Aware Decoding with Minimum Bayes Risk](https://mlsanity.com/paper/2503.05318/) (2025)
- [R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents](https://mlsanity.com/paper/2303.00732/) (2023)
- [Socratic Reasoning Improves Positive Text Rewriting](https://mlsanity.com/paper/2403.03029/) (2024)
- [DeepReviewer 2.0: A Traceable Agentic System for Auditable Scientific Peer Review](https://mlsanity.com/paper/2604.09590/) (2026)
- [Improving LoRA with Variational Learning](https://mlsanity.com/paper/2506.14280/) (2025)
- [Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue](https://mlsanity.com/paper/2508.05283/) (2025)
- [MathEDU: Feedback Generation on Problem-Solving Processes for Mathematical Learning Support](https://mlsanity.com/paper/2505.18056/) (2025)

## Built on by (most cited first, 1 of 1)

- [Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study](https://mlsanity.com/paper/2609.07395/) (2026)

## Abstract (arXiv)

With rapidly improving capabilities, Large Language Models (LLMs) are increasingly used in many complex real-world tasks. Beyond requiring in-depth knowledge and reasoning skills, many of these tasks exhibit a high degree of subjectivity and require that the outputs of the model can be trusted. While a lot of progress has been made to train better models, decision-making algorithms have received less attention. In this work, we present and evaluate various uncertainty-aware decision-making algorithms based on Bayesian decision theory and risk-averse decision making on the tasks of tutoring and automatic peer reviewing. Concretely, we take uncertainty over tutoring strategies and review scores into account when generating a tutor response or review and use conformal prediction to provide guarantees over strategy and score. We find empirically that these algorithms can improve the utility of the generations but need to be carefully implemented when ambiguity is high. For example, risk-averse rules can degrade performance by optimizing for generic outputs, while Bayesian methods tend to perform better. Our work uses techniques from decision theory to improve LLM-based decision-making and outlines open challenges for the community.