# On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbots

arXiv: https://arxiv.org/abs/2406.01633 · HTML: https://mlsanity.com/paper/2406.01633/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Builds on (most cited first, 23 of 23)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [A Long Way to Go: Investigating Length Correlations in RLHF](https://mlsanity.com/paper/2310.03716/) (2023)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [A Categorical Archive of ChatGPT Failures](https://mlsanity.com/paper/2302.03494/) (2023)
- [Active Prompting with Chain-of-Thought for Large Language Models](https://mlsanity.com/paper/2302.12246/) (2023)
- [Large Language Models as Zero-Shot Conversational Recommenders](https://mlsanity.com/paper/2308.10053/) (2023)
- [Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information](https://mlsanity.com/paper/1805.04655/) (2018)
- [The Metacognitive Demands and Opportunities of Generative AI](https://mlsanity.com/paper/2312.10893/) (2023)
- [Query expansion techniques for information retrieval: A survey](https://mlsanity.com/paper/1708.00247/) (2017)
- [Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledge](https://mlsanity.com/paper/2104.06828/) (2021)
- [LLF-Bench: Benchmark for Interactive Learning from Language Feedback](https://mlsanity.com/paper/2312.06853/) (2023)
- [Decision-Oriented Dialogue for Human-AI Collaboration](https://mlsanity.com/paper/2305.20076/) (2023)
- [Hindsight Learning for MDPs with Exogenous Inputs](https://mlsanity.com/paper/2207.06272/) (2022)
- [Analyzing and Learning from User Interactions for Search Clarification](https://mlsanity.com/paper/2006.00166/) (2020)
- [Interpretable User Satisfaction Estimation for Conversational Systems with Large Language Models](https://mlsanity.com/paper/2403.12388/) (2024)
- [Hierarchical Conversational Preference Elicitation with Bandit Feedback](https://mlsanity.com/paper/2209.06129/) (2022)

## Built on by (most cited first, 24 of 24)

- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [Flipping the Dialogue: Training and Evaluating User Language Models](https://mlsanity.com/paper/2510.06552/) (2025)
- [Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents](https://mlsanity.com/paper/2602.16699/) (2026)
- [Contextualized Evaluations: Judging Language Model Responses to Underspecified Queries](https://mlsanity.com/paper/2411.07237/) (2024)
- [Quantifying the Utility of User Simulators for Building Collaborative LLM Assistants](https://mlsanity.com/paper/2605.09808/) (2026)
- [Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents](https://mlsanity.com/paper/2605.12894/) (2026)
- [LLMs Get Lost in Evolving User Intent](https://mlsanity.com/paper/2607.20734/) (2026)
- [Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation](https://mlsanity.com/paper/2602.07338/) (2026)
- [LLMs Corrupt Your Documents When You Delegate](https://mlsanity.com/paper/2604.15597/) (2026)
- [Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards](https://mlsanity.com/paper/2510.18731/) (2025)
- [Test-Time Policy Adaptation for Enhanced Multi-Turn Interactions with LLMs](https://mlsanity.com/paper/2509.23166/) (2025)
- [ICPO: Illocution-Calibrated Policy Optimization for Multi-Turn Conversation](https://mlsanity.com/paper/2601.15330/) (2026)
- [Found in Conversation: LLMs Teach Themselves to Close the Multi-Turn Gap](https://mlsanity.com/paper/2605.24432/) (2026)
- [Pretraining with random noise for uncertainty calibration](https://mlsanity.com/paper/2412.17411/) (2024)
- [Who is the richest club in the championship? Detecting and Rewriting Underspecified Questions Improve QA Performance](https://mlsanity.com/paper/2602.11938/) (2026)
- [SeDT: Sentence-Transformer Decision-Transformer Conditioning for Multi-Turn Conversation Reliability](https://mlsanity.com/paper/2605.26788/) (2026)
- [Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging](https://mlsanity.com/paper/2607.10428/) (2026)
- [AcCoRD: Evaluating User-Agent Collaboration Under Realistic User Preference Dynamics](https://mlsanity.com/paper/2608.27818/) (2026)
- [Clarification Is Not Enough: Post-Clarification Answering Remains the Bottleneck in Multi-Turn QA](https://mlsanity.com/paper/2605.25204/) (2026)
- [Localizing Input Uncertainty Quantification for Large Language Models via Shapley Values](https://mlsanity.com/paper/2605.28170/) (2026)
- [AInterviewer: A Platform for Designing and Conducting AI-led Qualitative Interviews](https://mlsanity.com/paper/2606.20588/) (2026)
- [CITBench: A Comprehensive Benchmark for Interactive Tabular Data Processing with LLMs](https://mlsanity.com/paper/2608.00018/) (2026)
- [Lost in Conversation or Lost in Translation? Diagnosing Multi-Turn Degradation in RAG](https://mlsanity.com/paper/2609.36700/) (2026)
- [Collaborative Theory of Mind: Learning to Self-Revise by Anticipating User Responses](https://mlsanity.com/paper/2610.12872/) (2026)

## Abstract (arXiv)

We explore the use of Large Language Model (LLM-based) chatbots to power recommender systems. We observe that the chatbots respond poorly when they encounter under-specified requests (e.g., they make incorrect assumptions, hedge with a long response, or refuse to answer). We conjecture that such miscalibrated response tendencies (i.e., conversational priors) can be attributed to LLM fine-tuning using annotators -- single-turn annotations may not capture multi-turn conversation utility, and the annotators' preferences may not even be representative of users interacting with a recommender system. We first analyze public LLM chat logs to conclude that query under-specification is common. Next, we study synthetic recommendation problems with configurable latent item utilities and frame them as Partially Observed Decision Processes (PODP). We find that pre-trained LLMs can be sub-optimal for PODPs and derive better policies that clarify under-specified queries when appropriate. Then, we re-calibrate LLMs by prompting them with learned control messages to approximate the improved policy. Finally, we show empirically that our lightweight learning approach effectively uses logged conversation data to re-calibrate the response strategies of LLM-based chatbots for recommendation tasks.