# A Conditional Generative Chatbot using Transformer Model

arXiv: https://arxiv.org/abs/2306.02074 · HTML: https://mlsanity.com/paper/2306.02074/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 51 references that are in the map.
- First posted on arXiv 3 Jun 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues](https://mlsanity.com/paper/1605.06069/) (2016)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://mlsanity.com/paper/1506.08909/) (2015)
- [GODEL: Large-Scale Pre-Training for Goal-Directed Dialog](https://mlsanity.com/paper/2206.11309/) (2022)
- [Retrieval-Enhanced Adversarial Training for Neural Response Generation](https://mlsanity.com/paper/1809.04276/) (2018)
- [Neural Personalized Response Generation as Domain Adaptation](https://mlsanity.com/paper/1701.02073/) (2017)
- [Ensemble-Based Deep Reinforcement Learning for Chatbots](https://mlsanity.com/paper/1908.10422/) (2019)
- [A Financial Service Chatbot based on Deep Bidirectional Transformers](https://mlsanity.com/paper/2003.04987/) (2020)
- [Proactive Retrieval-based Chatbots based on Relevant Knowledge and Goals](https://mlsanity.com/paper/2107.08329/) (2021)
- [Improving Conditional Sequence Generative Adversarial Networks by Stepwise Evaluation](https://mlsanity.com/paper/1808.05599/) (2018)
- [EnsembleGAN: Adversarial Learning for Retrieval-Generation Ensemble Model on Short-Text Conversation](https://mlsanity.com/paper/2004.14592/) (2020)
- [Transformer Based Bengali Chatbot Using General Knowledge Dataset](https://mlsanity.com/paper/2111.03937/) (2021)

## Abstract (arXiv)

A Chatbot serves as a communication tool between a human user and a machine to achieve an appropriate answer based on the human input. In more recent approaches, a combination of Natural Language Processing and sequential models are used to build a generative Chatbot. The main challenge of these models is their sequential nature, which leads to less accurate results. To tackle this challenge, in this paper, a novel architecture is proposed using conditional Wasserstein Generative Adversarial Networks and a transformer model for answer generation in Chatbots. While the generator of the proposed model consists of a full transformer model to generate an answer, the discriminator includes only the encoder part of a transformer model followed by a classifier. To the best of our knowledge, this is the first time that a generative Chatbot is proposed using the embedded transformer in both generator and discriminator models. Relying on the parallel computing of the transformer model, the results of the proposed model on the Cornell Movie-Dialog corpus and the Chit-Chat datasets confirm the superiority of the proposed model compared to state-of-the-art alternatives using different evaluation metrics.