# Contextual Bandit Applications in Customer Support Bot

arXiv: https://arxiv.org/abs/2112.03210 · HTML: https://mlsanity.com/paper/2112.03210/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 20 references that are in the map.
- First posted on arXiv 6 Dec 2021.

## Builds on (most cited first, 8 of 8)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Recommendations as Treatments: Debiasing Learning and Evaluation](https://mlsanity.com/paper/1602.05352/) (2016)
- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling](https://mlsanity.com/paper/1802.09127/) (2018)
- [Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring](https://mlsanity.com/paper/1905.01969/) (2019)
- [Off-policy evaluation for slate recommendation](https://mlsanity.com/paper/1605.04812/) (2016)
- [Lessons from Contextual Bandit Learning in a Customer Support Bot](https://mlsanity.com/paper/1905.02219/) (2019)

## Built on by (most cited first, 7 of 7)

- [Neural Bandit with Arm Group Graph](https://mlsanity.com/paper/2206.03644/) (2022)
- [Extending Open Bandit Pipeline to Simulate Industry Challenges](https://mlsanity.com/paper/2209.04147/) (2022)
- [Learning in Distributed Contextual Linear Bandits Without Sharing the Context](https://mlsanity.com/paper/2206.04180/) (2022)
- [A Nonparametric Contextual Bandit with Arm-level Eligibility Control for Customer Service Routing](https://mlsanity.com/paper/2209.05278/) (2022)
- [SmartChoices: Augmenting Software with Learned Implementations](https://mlsanity.com/paper/2304.13033/) (2023)
- [Designing an Interpretable Interface for Contextual Bandits](https://mlsanity.com/paper/2409.15143/) (2024)
- [Bandit Simulation for Average Reward Inference](https://mlsanity.com/paper/2606.00913/) (2026)

## Abstract (arXiv)

Virtual support agents have grown in popularity as a way for businesses to provide better and more accessible customer service. Some challenges in this domain include ambiguous user queries as well as changing support topics and user behavior (non-stationarity). We do, however, have access to partial feedback provided by the user (clicks, surveys, and other events) which can be leveraged to improve the user experience. Adaptable learning techniques, like contextual bandits, are a natural fit for this problem setting. In this paper, we discuss real-world implementations of contextual bandits (CB) for the Microsoft virtual agent. It includes intent disambiguation based on neural-linear bandits (NLB) and contextual recommendations based on a collection of multi-armed bandits (MAB). Our solutions have been deployed to production and have improved key business metrics of the Microsoft virtual agent, as confirmed by A/B experiments. Results include a relative increase of over 12% in problem resolution rate and relative decrease of over 4% in escalations to a human operator. While our current use cases focus on intent disambiguation and contextual recommendation for support bots, we believe our methods can be extended to other domains.