# Generative Adversarial Imitation Learning for Empathy-based AI

arXiv: https://arxiv.org/abs/2105.13328 · HTML: https://mlsanity.com/paper/2105.13328/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 27 May 2021.

## Builds on (most cited first, 12 of 12)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [Deep Reinforcement Learning for Dialogue Generation](https://mlsanity.com/paper/1606.01541/) (2016)
- [TextGAIL: Generative Adversarial Imitation Learning for Text Generation](https://mlsanity.com/paper/2004.13796/) (2020)
- [CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning](https://mlsanity.com/paper/1911.03705/) (2019)
- [Improve Diverse Text Generation by Self Labeling Conditional Variational Auto Encoder](https://mlsanity.com/paper/1903.10842/) (2019)
- [Transformer-Based Neural Text Generation with Syntactic Guidance](https://mlsanity.com/paper/2010.01737/) (2020)

## Abstract (arXiv)

Generative adversarial imitation learning (GAIL) is a model-free algorithm that has been shown to provide strong results in imitating complex behaviors in high-dimensional environments. In this paper, we utilize the GAIL model for text generation to develop empathy-based context-aware conversational AI. Our model uses an expert trajectory of empathetic prompt-response dialogues which can accurately exhibit the correct empathetic emotion when generating a response. The Generator of the GAIL model uses the GPT-2 sequential pre-trained language model trained on 117 million parameters from 40 GB of internet data. We propose a novel application of an approach used in transfer learning to fine tune the GPT-2 model in order to generate concise, user-specific empathetic responses validated against the Discriminator. Our novel GAIL model utilizes a sentiment analysis history-based reinforcement learning approach to empathetically respond to human interactions in a personalized manner. We find that our model's response scores on various human-generated prompts collected from the Facebook Empathetic Dialogues dataset outperform baseline counterparts. Moreover, our model improves upon various history-based conversational AI models developed recently, as our model's performance over a sustained conversation of 3 or more interactions outperform similar conversational AI models.