# A comprehensive solution to retrieval-based chatbot construction

arXiv: https://arxiv.org/abs/2106.06139 · HTML: https://mlsanity.com/paper/2106.06139/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 11 Jun 2021.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://mlsanity.com/paper/1506.08909/) (2015)
- [Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots](https://mlsanity.com/paper/1612.01627/) (2016)
- [Modeling Multi-turn Conversation with Deep Utterance Aggregation](https://mlsanity.com/paper/1806.09102/) (2018)
- [Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots](https://mlsanity.com/paper/1805.02333/) (2018)
- [Improving Response Selection in Multi-Turn Dialogue Systems by Incorporating Domain Knowledge](https://mlsanity.com/paper/1809.03194/) (2018)
- [Sequential Neural Networks for Noetic End-to-End Response Selection](https://mlsanity.com/paper/2003.02126/) (2020)

## Built on by (most cited first, 2 of 2)

- [Deploying a Retrieval based Response Model for Task Oriented Dialogues](https://mlsanity.com/paper/2210.14379/) (2022)
- [Cognitively Inspired Components for Social Conversational Agents](https://mlsanity.com/paper/2311.05450/) (2023)

## Abstract (arXiv)

In this paper we present the results of our experiments in training and deploying a self-supervised retrieval-based chatbot trained with contrastive learning for assisting customer support agents. In contrast to most existing research papers in this area where the focus is on solving just one component of a deployable chatbot, we present an end-to-end set of solutions to take the reader from an unlabelled chatlogs to a deployed chatbot. This set of solutions includes creating a self-supervised dataset and a weakly labelled dataset from chatlogs, as well as a systematic approach to selecting a fixed list of canned responses. We present a hierarchical-based RNN architecture for the response selection model, chosen for its ability to cache intermediate utterance embeddings, which helped to meet deployment inference speed requirements. We compare the performance of this architecture across 3 different learning objectives: self-supervised contrastive learning, binary classification, and multi-class classification. We find that using a self-supervised contrastive learning model outperforms training the binary and multi-class classification models on a weakly labelled dataset. Our results validate that the self-supervised contrastive learning approach can be effectively used for a real-world chatbot scenario.