An Empirical Study of Multitask Learning to Improve Open Domain Dialogue Systems
Not on the map. Why?
Abstract (arXiv)
Autoregressive models used to generate responses in open-domain dialogue systems often struggle to take long-term context into account and to maintain consistency over a dialogue. Previous research in open-domain dialogue generation has shown that the use of auxiliary tasks can introduce inductive biases that encourage the model to improve these qualities. However, most previous research has focused on encoder-only or encoder/decoder models, while the use of auxiliary tasks in decoder-only autoregressive models is under-explored. This paper describes an investigation where four different auxiliary tasks are added to small and medium-sized GPT-2 models fine-tuned on the PersonaChat and DailyDialog datasets. The results show that the introduction of the new auxiliary tasks leads to small but consistent improvement in evaluations of the investigated models.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 34 references that are in the map. First posted on arXiv 17 Apr 2023.
Builds on
27 of 34 references.
References · 27
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
- 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
- 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
- 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
- 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
- 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
- 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
- 2020Towards a Human-like Open-Domain Chatbotcited by 467
- 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
15 more
- 2016A Persona-Based Neural Conversation Modelcited by 221
- 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
- 2015A Neural Conversational Modelcited by 195
- 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
- 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 178
- 2016A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialoguescited by 174
- 2018Neural Approaches to Conversational AIcited by 164
- 2017A Knowledge-Grounded Neural Conversation Modelcited by 136
- 2019Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systemscited by 110
- 2019Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Studycited by 47
- 2019BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformercited by 37
- 2020DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterancescited by 36
- 2019Pretraining Methods for Dialog Context Representation Learningcited by 35
- 2020Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Taskscited by 14
- 2017Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Modelscited by 12
Built on by
Citing papers per year: 2026: 1.
1 paper.