# EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems

arXiv: https://arxiv.org/abs/2003.02958 · HTML: https://mlsanity.com/paper/2003.02958/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Mar 2020.

## Built on by (most cited first, 19 of 19)

- [Towards Emotional Support Dialog Systems](https://mlsanity.com/paper/2106.01144/) (2021)
- [Building Emotional Support Chatbots in the Era of LLMs](https://mlsanity.com/paper/2308.11584/) (2023)
- [CoMAE: A Multi-factor Hierarchical Framework for Empathetic Response Generation](https://mlsanity.com/paper/2105.08316/) (2021)
- [CoMPM: Context Modeling with Speaker's Pre-trained Memory Tracking for Emotion Recognition in Conversation](https://mlsanity.com/paper/2108.11626/) (2021)
- [CPED: A Large-Scale Chinese Personalized and Emotional Dialogue Dataset for Conversational AI](https://mlsanity.com/paper/2205.14727/) (2022)
- [Multi-Task Learning of Generation and Classification for Emotion-Aware Dialogue Response Generation](https://mlsanity.com/paper/2105.11696/) (2021)
- [Automatically Select Emotion for Response via Personality-affected Emotion Transition](https://mlsanity.com/paper/2106.15846/) (2021)
- [Empathetic Conversational Systems: A Review of Current Advances, Gaps, and Opportunities](https://mlsanity.com/paper/2206.05017/) (2022)
- [Empathetic Dialogue Generation with Pre-trained RoBERTa-GPT2 and External Knowledge](https://mlsanity.com/paper/2109.03004/) (2021)
- [Mitigating Toxic Degeneration with Empathetic Data: Exploring the Relationship Between Toxicity and Empathy](https://mlsanity.com/paper/2205.07233/) (2022)
- [DiagESC: Dialogue Synthesis for Integrating Depression Diagnosis into Emotional Support Conversation](https://mlsanity.com/paper/2408.06044/) (2024)
- [Emotion-Aware Transformer Encoder for Empathetic Dialogue Generation](https://mlsanity.com/paper/2204.11320/) (2022)
- [Towards Detecting Need for Empathetic Response in Motivational Interviewing](https://mlsanity.com/paper/2105.09649/) (2021)
- [Personality-affected Emotion Generation in Dialog Systems](https://mlsanity.com/paper/2404.07229/) (2024)
- [ConceptNet infused DialoGPT for Underlying Commonsense Understanding and Reasoning in Dialogue Response Generation](https://mlsanity.com/paper/2209.15109/) (2022)
- [CloneBot: Personalized Dialogue-Response Predictions](https://mlsanity.com/paper/2103.16750/) (2021)
- [Estimating Subjective Crowd-Evaluations as an Additional Objective to Improve Natural Language Generation](https://mlsanity.com/paper/2104.05224/) (2021)
- [Empathetic Response Generation with State Management](https://mlsanity.com/paper/2205.03676/) (2022)
- [QNet: A Quantum-native Sequence Encoder Architecture](https://mlsanity.com/paper/2210.17262/) (2022)

## Abstract (arXiv)

Understanding emotions and responding accordingly is one of the biggest challenges of dialog systems. This paper presents EmpTransfo, a multi-head Transformer architecture for creating an empathetic dialog system. EmpTransfo utilizes state-of-the-art pre-trained models (e.g., OpenAI-GPT) for language generation, though models with different sizes can be used. We show that utilizing the history of emotions and other metadata can improve the quality of generated conversations by the dialog system. Our experimental results using a challenging language corpus show that the proposed approach outperforms other models in terms of Hit@1 and PPL (Perplexity).