A Knowledge-Grounded Dialog System Based on Pre-Trained Language Models
Not on the map. Why?
Abstract (arXiv)
We present a knowledge-grounded dialog system developed for the ninth Dialog System Technology Challenge (DSTC9) Track 1 - Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access. We leverage transfer learning with existing language models to accomplish the tasks in this challenge track. Specifically, we divided the task into four sub-tasks and fine-tuned several Transformer models on each of the sub-tasks. We made additional changes that yielded gains in both performance and efficiency, including the combination of the model with traditional entity-matching techniques, and the addition of a pointer network to the output layer of the language model.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 11 of its 18 references that are in the map. First posted on arXiv 28 Jun 2021.
Builds on
11 of 18 references.
References · 11
- 2017Attention Is All You Needcited by 60,361
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
- 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
- 2015Pointer Networkscited by 895
- 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
- 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
- 2020Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Accesscited by 58
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.