A Unified Pre-training Framework for Conversational AI
Not on the map. Why?
Abstract (arXiv)
In this work, we explore the application of PLATO-2 on various dialogue systems, including open-domain conversation, knowledge grounded dialogue, and task-oriented conversation. PLATO-2 is initially designed as an open-domain chatbot, trained via two-stage curriculum learning. In the first stage, a coarse-grained response generation model is learned to fit the simplified one-to-one mapping relationship. This model is applied to the task-oriented conversation, given that the semantic mappings tend to be deterministic in task completion. In the second stage, another fine-grained generation model and an evaluation model are further learned for diverse response generation and coherence estimation, respectively. With superior capability on capturing one-to-many mapping, such models are suitable for the open-domain conversation and knowledge grounded dialogue. For the comprehensive evaluation of PLATO-2, we have participated in multiple tasks of DSTC9, including interactive evaluation of open-domain conversation (Track3-task2), static evaluation of knowledge grounded dialogue (Track3-task1), and end-to-end task-oriented conversation (Track2-task1). PLATO-2 has obtained the 1st place in all three tasks, verifying its effectiveness as a unified framework for various dialogue systems.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 24 of its 32 references that are in the map. First posted on arXiv 6 May 2021.
Builds on
24 of 32 references.
References · 24
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
- 2018Hierarchical Neural Story Generationcited by 1,182
- 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
- 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
- 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
- 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
- 2020Recipes for building an open-domain chatbotcited by 593
- 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
12 more
- 2020Towards a Human-like Open-Domain Chatbotcited by 467
- 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
- 2017Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoderscited by 222
- 2020Recipes for Safety in Open-domain Chatbotscited by 168
- 2018Neural Approaches to Conversational AIcited by 164
- 2019PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variablecited by 145
- 2015Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systemscited by 134
- 2020Sequential Latent Knowledge Selection for Knowledge-Grounded Dialoguecited by 82
- 2020PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learningcited by 79
- 2020SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teachingcited by 71
- 2020ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systemscited by 52
- 2016DialPort: Connecting the Spoken Dialog Research Community to Real User Datacited by 5
Built on by
Citing papers per year: 2021: 1; 2022: 0; 2023: 1.
2 papers.