# A Literature Survey of Recent Advances in Chatbots

arXiv: https://arxiv.org/abs/2201.06657 · HTML: https://mlsanity.com/paper/2201.06657/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 106 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [Deep Reinforcement Learning for Dialogue Generation](https://mlsanity.com/paper/1606.01541/) (2016)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [A Neural Conversational Model](https://mlsanity.com/paper/1506.05869/) (2015)
- [The Evolved Transformer](https://mlsanity.com/paper/1901.11117/) (2019)
- [The Design and Implementation of XiaoIce, an Empathetic Social Chatbot](https://mlsanity.com/paper/1812.08989/) (2018)
- [Recipes for Safety in Open-domain Chatbots](https://mlsanity.com/paper/2010.07079/) (2020)
- [Neural Responding Machine for Short-Text Conversation](https://mlsanity.com/paper/1503.02364/) (2015)
- [Adversarial Learning for Neural Dialogue Generation](https://mlsanity.com/paper/1701.06547/) (2017)
- [A Neural Network Approach to Context-Sensitive Generation of Conversational Responses](https://mlsanity.com/paper/1506.06714/) (2015)
- [From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots](https://mlsanity.com/paper/1801.01957/) (2018)
- [ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons](https://mlsanity.com/paper/1909.03087/) (2019)
- [Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems](https://mlsanity.com/paper/1906.09308/) (2019)
- [XPersona: Evaluating Multilingual Personalized Chatbot](https://mlsanity.com/paper/2003.07568/) (2020)
- [An Affect-Rich Neural Conversational Model with Biased Attention and Weighted Cross-Entropy Loss](https://mlsanity.com/paper/1811.07078/) (2018)
- [Evaluating Quality of Chatbots and Intelligent Conversational Agents](https://mlsanity.com/paper/1704.04579/) (2017)
- [Adversarial Evaluation of Dialogue Models](https://mlsanity.com/paper/1701.08198/) (2017)
- [Ensemble-Based Deep Reinforcement Learning for Chatbots](https://mlsanity.com/paper/1908.10422/) (2019)
- [Exploring Context-Aware Conversational Agents in Software Development](https://mlsanity.com/paper/2006.02370/) (2020)
- [Seq2Seq AI Chatbot with Attention Mechanism](https://mlsanity.com/paper/2006.02767/) (2020)

## Built on by (most cited first, 26 of 26)

- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [Bias and Fairness in Chatbots: An Overview](https://mlsanity.com/paper/2309.08836/) (2023)
- [Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systems](https://mlsanity.com/paper/2502.00409/) (2025)
- [Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue Response Generation Models by Causal Discovery](https://mlsanity.com/paper/2303.01962/) (2023)
- [Evaluating Task-oriented Dialogue Systems: A Systematic Review of Measures, Constructs and their Operationalisations](https://mlsanity.com/paper/2312.13871/) (2023)
- [Deceptive AI Ecosystems: The Case of ChatGPT](https://mlsanity.com/paper/2306.13671/) (2023)
- [A Comparison of LLM Finetuning Methods & Evaluation Metrics with Travel Chatbot Use Case](https://mlsanity.com/paper/2408.03562/) (2024)
- [State-of-the-art in Open-domain Conversational AI: A Survey](https://mlsanity.com/paper/2205.00965/) (2022)
- [Computational Argumentation-based Chatbots: a Survey](https://mlsanity.com/paper/2401.03454/) (2024)
- [Beyond Traditional Teaching: The Potential of Large Language Models and Chatbots in Graduate Engineering Education](https://mlsanity.com/paper/2309.13059/) (2023)
- [Ultra-Long Sequence Distributed Transformer](https://mlsanity.com/paper/2311.02382/) (2023)
- [Impacts of Anthropomorphizing Large Language Models in Learning Environments](https://mlsanity.com/paper/2408.03945/) (2024)
- [Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2505.21907/) (2025)
- [Cases of EFL Secondary Students' Prompt Engineering Pathways to Complete a Writing Task with ChatGPT](https://mlsanity.com/paper/2307.05493/) (2023)
- [KoDialogBench: Evaluating Conversational Understanding of Language Models with Korean Dialogue Benchmark](https://mlsanity.com/paper/2402.17377/) (2024)
- [The Chai Platform's AI Safety Framework](https://mlsanity.com/paper/2306.02979/) (2023)
- [Modular Conversational Agents for Surveys and Interviews](https://mlsanity.com/paper/2412.17049/) (2024)
- [Finding A Voice: Exploring the Potential of African American Dialect and Voice Generation for Chatbots](https://mlsanity.com/paper/2501.03441/) (2025)
- [A Proposal for Evaluating the Operational Risk for ChatBots based on Large Language Models](https://mlsanity.com/paper/2505.04784/) (2025)
- [Implementing a Chatbot Solution for Learning Management System](https://mlsanity.com/paper/2206.13187/) (2022)
- [A Reference Model for Collaborative Business Intelligence Virtual Assistants](https://mlsanity.com/paper/2304.10556/) (2023)
- [Creation Of A ChatBot Based On Natural Language Proccesing For Whatsapp](https://mlsanity.com/paper/2310.10675/) (2023)
- [Collaborative business intelligence virtual assistant](https://mlsanity.com/paper/2312.12778/) (2023)
- [Intelligent Co-Design: An Interactive LLM Framework for Interior Spatial Design via Multi-Modal Agents](https://mlsanity.com/paper/2603.15341/) (2026)
- [Large Language Models for Token-Efficient and Semantic-Preserving Opinion Summarization](https://mlsanity.com/paper/2607.10825/) (2026)

## Abstract (arXiv)

Chatbots are intelligent conversational computer systems designed to mimic human conversation to enable automated online guidance and support. The increased benefits of chatbots led to their wide adoption by many industries in order to provide virtual assistance to customers. Chatbots utilise methods and algorithms from two Artificial Intelligence domains: Natural Language Processing and Machine Learning. However, there are many challenges and limitations in their application. In this survey we review recent advances on chatbots, where Artificial Intelligence and Natural Language processing are used. We highlight the main challenges and limitations of current work and make recommendations for future research investigation.