Unified Knowledge Prompt Pre-training for Customer Service Dialogues
Not on the map. Why?
Abstract (arXiv)
Dialogue bots have been widely applied in customer service scenarios to provide timely and user-friendly experience. These bots must classify the appropriate domain of a dialogue, understand the intent of users, and generate proper responses. Existing dialogue pre-training models are designed only for several dialogue tasks and ignore weakly-supervised expert knowledge in customer service dialogues. In this paper, we propose a novel unified knowledge prompt pre-training framework, UFA (Unified Model For All Tasks), for customer service dialogues. We formulate all the tasks of customer service dialogues as a unified text-to-text generation task and introduce a knowledge-driven prompt strategy to jointly learn from a mixture of distinct dialogue tasks. We pre-train UFA on a large-scale Chinese customer service corpus collected from practical scenarios and get significant improvements on both natural language understanding (NLU) and natural language generation (NLG) benchmarks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 18 of its 26 references that are in the map. First posted on arXiv 31 Aug 2022.
Builds on
18 of 26 references.
References · 18
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
- 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
- 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
- 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
- 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 262
- 2020TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialoguecited by 185
6 more
- 2019ConveRT: Efficient and Accurate Conversational Representations from Transformerscited by 95
- 2021DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarizationcited by 72
- 2020DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterancescited by 36
- 2021Structural Pre-training for Dialogue Comprehensioncited by 20
- 2022Domain-Oriented Prefix-Tuning: Towards Efficient and Generalizable Fine-tuning for Zero-Shot Dialogue Summarizationcited by 14
- 2021DialogueBERT: A Self-Supervised Learning based Dialogue Pre-training Encodercited by 11
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.