# Ask to Be Sure: Informative Interactions for Confident Multi-Turn LLM Recommendation

arXiv: https://arxiv.org/abs/2608.15949 · HTML: https://mlsanity.com/paper/2608.15949/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 16 Aug 2026.

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL](https://mlsanity.com/paper/2402.19446/) (2024)
- [Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation](https://mlsanity.com/paper/2311.09049/) (2023)
- [Towards Deep Conversational Recommendations](https://mlsanity.com/paper/1812.07617/) (2018)
- [Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs](https://mlsanity.com/paper/2502.09597/) (2025)
- [Large Language Models as Zero-Shot Conversational Recommenders](https://mlsanity.com/paper/2308.10053/) (2023)
- [CollabLLM: From Passive Responders to Active Collaborators](https://mlsanity.com/paper/2502.00640/) (2025)
- [Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion](https://mlsanity.com/paper/2007.04032/) (2020)
- [Conversational Recommender System](https://mlsanity.com/paper/1806.03277/) (2018)
- [Towards Knowledge-Based Recommender Dialog System](https://mlsanity.com/paper/1908.05391/) (2019)
- [INSPIRED: Toward Sociable Recommendation Dialog Systems](https://mlsanity.com/paper/2009.14306/) (2020)
- [Multi-turn Reinforcement Learning from Preference Human Feedback](https://mlsanity.com/paper/2405.14655/) (2024)
- [Collaborative Large Language Model for Recommender Systems](https://mlsanity.com/paper/2311.01343/) (2023)
- [Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning](https://mlsanity.com/paper/2105.09710/) (2021)
- [Regressing the Relative Future: Efficient Policy Optimization for Multi-turn RLHF](https://mlsanity.com/paper/2410.04612/) (2024)
- [Collaborative Retrieval for Large Language Model-based Conversational Recommender Systems](https://mlsanity.com/paper/2502.14137/) (2025)
- [Taxonomy-Guided Zero-Shot Recommendations with LLMs](https://mlsanity.com/paper/2406.14043/) (2024)
- [Uncertainty Quantification and Decomposition for LLM-based Recommendation](https://mlsanity.com/paper/2501.17630/) (2025)
- [SAPIENT: Mastering Multi-turn Conversational Recommendation with Strategic Planning and Monte Carlo Tree Search](https://mlsanity.com/paper/2410.09580/) (2024)

## Abstract (arXiv)

Recent advances in large language models (LLMs) have enabled their use as conversational recommender systems (CRS), demonstrating strong recommendation accuracy and natural dialogue. However, guiding multi-turn interactions to elicit user preferences effectively remains challenging. Existing approaches either use separate reinforcement learning agents with templated interactions or optimize for interactivity judged by another LLM, without measuring how much useful information is actually gained. We propose a new approach that quantifies the effectiveness of each interaction by the reduction in the assistant's uncertainty, measured via entropy over recommendations. We apply this entropy reduction as a reward---without relying on ground-truth recommendations, which are often unavailable in real-world scenarios---to fine-tune the LLM, enabling strategic interaction generation. Empirical results with supervised fine-tuning (SFT) and direct preference optimization (DPO) on the INSPIRED and ReDial datasets show that our method improves both recommendation quality and conversational efficiency.