mlsanityData to 7 Oct 2026

Empathetic Dialogue Generation with Pre-trained RoBERTa-GPT2 and External Knowledge

arXiv 2109.03004 ↗First posted 7 Sep 2021cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

One challenge for dialogue agents is to recognize feelings of the conversation partner and respond accordingly. In this work, RoBERTa-GPT2 is proposed for empathetic dialogue generation, where the pre-trained auto-encoding RoBERTa is utilised as encoder and the pre-trained auto-regressive GPT-2 as decoder. With the combination of the pre-trained RoBERTa and GPT-2, our model realizes a new state-of-the-art emotion accuracy. To enable the empathetic ability of RoBERTa-GPT2 model, we propose a commonsense knowledge and emotional concepts extractor, in which the commonsensible and emotional concepts of dialogue context are extracted for the GPT-2 decoder. The experiment results demonstrate that the empathetic dialogue generation benefits from both pre-trained encoder-decoder architecture and external knowledge.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 33 of its 50 references that are in the map. First posted on arXiv 7 Sep 2021.

Builds on

33 of 50 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2018Hierarchical Neural Story Generationcited by 1,182
  8. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  9. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  10. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  11. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  12. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
21 more
  1. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 359
  2. 2016A Persona-Based Neural Conversation Modelcited by 221
  3. 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
  4. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  5. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 178
  6. 2017Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memorycited by 167
  7. 2019Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systemscited by 110
  8. 2019Knowledge-Enriched Transformer for Emotion Detection in Textual Conversationscited by 92
  9. 2019MoEL: Mixture of Empathetic Listenerscited by 91
  10. 2020Few-shot Natural Language Generation for Task-Oriented Dialogcited by 89
  11. 2018EmotionLines: An Emotion Corpus of Multi-Party Conversationscited by 80
  12. 2017MojiTalk: Generating Emotional Responses at Scalecited by 64
  13. 2019Affect-Driven Dialog Generationcited by 50
  14. 2020CDL: Curriculum Dual Learning for Emotion-Controllable Response Generationcited by 43
  15. 2021Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interactioncited by 21
  16. 2020EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systemscited by 19
  17. 2017Augmenting End-to-End Dialog Systems with Commonsense Knowledgecited by 18
  18. 2018Towards Empathetic Open-domain Conversation Models: a New Benchmark and Datasetcited by 18
  19. 2016Generative Deep Neural Networks for Dialogue: A Short Reviewcited by 13
  20. 2018A Syntactically Constrained Bidirectional-Asynchronous Approach for Emotional Conversation Generationcited by 11
  21. 2021Empathetic BERT2BERT Conversational Model: Learning Arabic Language Generation with Little Datacited by 7

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 3; 2024: 2.

8 papers.

  1. 2022Empathetic Conversational Systems: A Review of Current Advances, Gaps, and Opportunitiescited by 14
  2. 2023Knowledge-enhanced Memory Model for Emotional Support Conversationcited by 7
  3. 2024A Review of Human Emotion Synthesis Based on Generative Technologycited by 7
  4. 2023CAB: Empathetic Dialogue Generation with Cognition, Affection and Behaviorcited by 6
  5. 2022Acoustic Modeling for End-to-End Empathetic Dialogue Speech Synthesis Using Linguistic and Prosodic Contexts of Dialogue Historycited by 3
  6. 2024Acknowledgment of Emotional States: Generating Validating Responses for Empathetic Dialoguecited by 3
  7. 2023Unified Conversational Models with System-Initiated Transitions between Chit-Chat and Task-Oriented Dialoguescited by 2
  8. 2022ConceptNet infused DialoGPT for Underlying Commonsense Understanding and Reasoning in Dialogue Response Generationcited by 1