# Lessons from Contextual Bandit Learning in a Customer Support Bot

arXiv: https://arxiv.org/abs/1905.02219 · HTML: https://mlsanity.com/paper/1905.02219/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2019.

## Built on by (most cited first, 5 of 5)

- [Contextual Bandit Applications in Customer Support Bot](https://mlsanity.com/paper/2112.03210/) (2021)
- [Scalable and Robust Self-Learning for Skill Routing in Large-Scale Conversational AI Systems](https://mlsanity.com/paper/2204.07135/) (2022)
- [Constrained Policy Optimization for Controlled Self-Learning in Conversational AI Systems](https://mlsanity.com/paper/2209.08429/) (2022)
- [Scalable and Safe Remediation of Defective Actions in Self-Learning Conversational Systems](https://mlsanity.com/paper/2305.10528/) (2023)
- [Dynamic Governance of Multi-LLM Agent Systems for Collaborative Conversational Outcomes](https://mlsanity.com/paper/2608.11207/) (2026)

## Abstract (arXiv)

In this work, we describe practical lessons we have learned from successfully using contextual bandits (CBs) to improve key business metrics of the Microsoft Virtual Agent for customer support. While our current use cases focus on single step einforcement learning (RL) and mostly in the domain of natural language processing and information retrieval we believe many of our findings are generally applicable. Through this article, we highlight certain issues that RL practitioners may encounter in similar types of applications as well as offer practical solutions to these challenges.