# Customized Nonlinear Bandits for Online Response Selection in Neural Conversation Models

arXiv: https://arxiv.org/abs/1711.08493 · HTML: https://mlsanity.com/paper/1711.08493/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2017.

## Built on by (most cited first, 6 of 6)

- [HD-CB: The First Exploration of Hyperdimensional Computing for Contextual Bandits Problems](https://mlsanity.com/paper/2501.16863/) (2025)
- [An Efficient Algorithm for Deep Stochastic Contextual Bandits](https://mlsanity.com/paper/2104.05613/) (2021)
- [Etat de l'art sur l'application des bandits multi-bras](https://mlsanity.com/paper/2101.00001/) (2021)
- [Information Cascade Prediction under Public Emergencies: A Survey](https://mlsanity.com/paper/2404.01319/) (2024)
- [Optimality of Sub-network Laplace Approximations: New Results and Methods](https://mlsanity.com/paper/2605.09075/) (2026)
- [User-Aware Active Knowledge Acquisition for Emotional Support Dialogue](https://mlsanity.com/paper/2605.29715/) (2026)

## Abstract (arXiv)

Dialog response selection is an important step towards natural response generation in conversational agents. Existing work on neural conversational models mainly focuses on offline supervised learning using a large set of context-response pairs. In this paper, we focus on online learning of response selection in retrieval-based dialog systems. We propose a contextual multi-armed bandit model with a nonlinear reward function that uses distributed representation of text for online response selection. A bidirectional LSTM is used to produce the distributed representations of dialog context and responses, which serve as the input to a contextual bandit. In learning the bandit, we propose a customized Thompson sampling method that is applied to a polynomial feature space in approximating the reward. Experimental results on the Ubuntu Dialogue Corpus demonstrate significant performance gains of the proposed method over conventional linear contextual bandits. Moreover, we report encouraging response selection performance of the proposed neural bandit model using the Recall@k metric for a small set of online training samples.