# SAPIENT: Mastering Multi-turn Conversational Recommendation with Strategic Planning and Monte Carlo Tree Search

arXiv: https://arxiv.org/abs/2410.09580 · HTML: https://mlsanity.com/paper/2410.09580/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 53 references that are in the map.
- First posted on arXiv 12 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction](https://mlsanity.com/paper/1906.00949/) (2019)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- [A Survey on Conversational Recommender Systems](https://mlsanity.com/paper/2004.00646/) (2020)
- [Large Language Models as Zero-Shot Conversational Recommenders](https://mlsanity.com/paper/2308.10053/) (2023)
- [Maxmin Q-learning: Controlling the Estimation Bias of Q-learning](https://mlsanity.com/paper/2002.06487/) (2020)
- [Conversational Recommender System](https://mlsanity.com/paper/1806.03277/) (2018)
- [Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems](https://mlsanity.com/paper/2002.09102/) (2020)
- [Interactive Path Reasoning on Graph for Conversational Recommendation](https://mlsanity.com/paper/2007.00194/) (2020)
- [Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planning](https://mlsanity.com/paper/2305.13660/) (2023)
- [Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning](https://mlsanity.com/paper/2105.09710/) (2021)
- [Evaluating Conversational Recommender Systems via User Simulation](https://mlsanity.com/paper/2006.08732/) (2020)
- [Planning Like Human: A Dual-process Framework for Dialogue Planning](https://mlsanity.com/paper/2406.05374/) (2024)
- [Large-scale Interactive Recommendation with Tree-structured Policy Gradient](https://mlsanity.com/paper/1811.05869/) (2018)
- [Multiple Choice Questions based Multi-Interest Policy Learning for Conversational Recommendation](https://mlsanity.com/paper/2112.11775/) (2021)
- [Quantifying and Mitigating Popularity Bias in Conversational Recommender Systems](https://mlsanity.com/paper/2208.03298/) (2022)
- [Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning](https://mlsanity.com/paper/2208.02294/) (2022)
- [Lending Interaction Wings to Recommender Systems with Conversational Agents](https://mlsanity.com/paper/2310.04230/) (2023)
- [Unintended Bias in Language Model-driven Conversational Recommendation](https://mlsanity.com/paper/2201.06224/) (2022)

## Built on by (most cited first, 6 of 6)

- [ChatShopBuddy: Towards Reliable Conversational Shopping Agents via Reinforcement Learning](https://mlsanity.com/paper/2603.06065/) (2026)
- [Ask to Be Sure: Informative Interactions for Confident Multi-Turn LLM Recommendation](https://mlsanity.com/paper/2608.15949/) (2026)
- [When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models](https://mlsanity.com/paper/2608.30395/) (2026)
- [Towards Effective Structured Context Modeling for Conversational Recommender Systems via Dual-node Monte Carlo Tree Search](https://mlsanity.com/paper/2609.00618/) (2026)
- [CC-Mediation: Evaluating Large Language Models for Cross-Cultural Conflict Mediation](https://mlsanity.com/paper/2609.04855/) (2026)
- [RealWorldShop: Benchmarking and Improving Conversational Shopping Agents in Real-World E-commerce](https://mlsanity.com/paper/2609.38974/) (2026)

## Abstract (arXiv)

Conversational Recommender Systems (CRS) proactively engage users in interactive dialogues to elicit user preferences and provide personalized recommendations. Existing methods train Reinforcement Learning (RL)-based agent with greedy action selection or sampling strategy, and may suffer from suboptimal conversational planning. To address this, we present a novel Monte Carlo Tree Search (MCTS)-based CRS framework SAPIENT. SAPIENT consists of a conversational agent (S-agent) and a conversational planner (S-planner). S-planner builds a conversational search tree with MCTS based on the initial actions proposed by S-agent to find conversation plans. The best conversation plans from S-planner are used to guide the training of S-agent, creating a self-training loop where S-agent can iteratively improve its capability for conversational planning. Furthermore, we propose an efficient variant SAPIENT for trade-off between training efficiency and performance. Extensive experiments on four benchmark datasets validate the effectiveness of our approach, showing that SAPIENT outperforms the state-of-the-art baselines. Our code and data are accessible through https://github.com/ninglab/SAPIENT.