# Low-Resource Adaptation of Open-Domain Generative Chatbots

arXiv: https://arxiv.org/abs/2108.06329 · HTML: https://mlsanity.com/paper/2108.06329/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 13 Aug 2021.

## Builds on (most cited first, 14 of 14)

- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [The Second Conversational Intelligence Challenge (ConvAI2)](https://mlsanity.com/paper/1902.00098/) (2019)
- [The Design and Implementation of XiaoIce, an Empathetic Social Chatbot](https://mlsanity.com/paper/1812.08989/) (2018)
- [Open-Domain Question Answering Goes Conversational via Question Rewriting](https://mlsanity.com/paper/2010.04898/) (2020)
- [Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skills](https://mlsanity.com/paper/2004.08449/) (2020)
- [Retrieve and Refine: Improved Sequence Generation Models For Dialogue](https://mlsanity.com/paper/1808.04776/) (2018)
- [A Deep Reinforcement Learning Chatbot](https://mlsanity.com/paper/1709.02349/) (2017)
- [Gunrock: A Social Bot for Complex and Engaging Long Conversations](https://mlsanity.com/paper/1910.03042/) (2019)
- [Natural Language Generation at Scale: A Case Study for Open Domain Question Answering](https://mlsanity.com/paper/1903.08097/) (2019)

## Built on by (most cited first, 2 of 2)

- [chatClimate: Grounding Conversational AI in Climate Science](https://mlsanity.com/paper/2304.05510/) (2023)
- [From Words and Exercises to Wellness: Farsi Chatbot for Self-Attachment Technique](https://mlsanity.com/paper/2310.09362/) (2023)

## Abstract (arXiv)

Recent work building open-domain chatbots has demonstrated that increasing model size improves performance. On the other hand, latency and connectivity considerations dictate the move of digital assistants on the device. Giving a digital assistant like Siri, Alexa, or Google Assistant the ability to discuss just about anything leads to the need for reducing the chatbot model size such that it fits on the user's device. We demonstrate that low parameter models can simultaneously retain their general knowledge conversational abilities while improving in a specific domain. Additionally, we propose a generic framework that accounts for variety in question types, tracks reference throughout multi-turn conversations, and removes inconsistent and potentially toxic responses. Our framework seamlessly transitions between chatting and performing transactional tasks, which will ultimately make interactions with digital assistants more human-like. We evaluate our framework on 1 internal and 4 public benchmark datasets using both automatic (Perplexity) and human (SSA - Sensibleness and Specificity Average) evaluation metrics and establish comparable performance while reducing model parameters by 90%.