# Learning to Select External Knowledge with Multi-Scale Negative Sampling

arXiv: https://arxiv.org/abs/2102.02096 · HTML: https://mlsanity.com/paper/2102.02096/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 3 Feb 2021.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders](https://mlsanity.com/paper/1703.10960/) (2017)
- [MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines](https://mlsanity.com/paper/2007.12720/) (2020)
- [PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable](https://mlsanity.com/paper/1910.07931/) (2019)
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1508.01745/) (2015)
- [Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2002.07510/) (2020)
- [Learning to Select Knowledge for Response Generation in Dialog Systems](https://mlsanity.com/paper/1902.04911/) (2019)
- [PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning](https://mlsanity.com/paper/2006.16779/) (2020)
- [Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access](https://mlsanity.com/paper/2006.03533/) (2020)
- [Augmenting Transformers with KNN-Based Composite Memory for Dialogue](https://mlsanity.com/paper/2004.12744/) (2020)
- [End-to-End Task-Completion Neural Dialogue Systems](https://mlsanity.com/paper/1703.01008/) (2017)
- [Robust Zero-Shot Cross-Domain Slot Filling with Example Values](https://mlsanity.com/paper/1906.06870/) (2019)
- [Schema-Guided Dialogue State Tracking Task at DSTC8](https://mlsanity.com/paper/2002.01359/) (2020)

## Built on by (most cited first, 16 of 16)

- [PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation](https://mlsanity.com/paper/2109.09519/) (2021)
- [TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue Modeling on Spoken Conversations](https://mlsanity.com/paper/2112.12441/) (2021)
- [GripRank: Bridging the Gap between Retrieval and Generation via the Generative Knowledge Improved Passage Ranking](https://mlsanity.com/paper/2305.18144/) (2023)
- [Should We Fine-Tune or RAG? Evaluating Different Techniques to Adapt LLMs for Dialogue](https://mlsanity.com/paper/2406.06399/) (2024)
- [OPERA: Harmonizing Task-Oriented Dialogs and Information Seeking Experience](https://mlsanity.com/paper/2206.12449/) (2022)
- [Task-oriented Document-Grounded Dialog Systems by HLTPR@RWTH for DSTC9 and DSTC10](https://mlsanity.com/paper/2304.07101/) (2023)
- [End-to-End Task-Oriented Dialog Modeling with Semi-Structured Knowledge Management](https://mlsanity.com/paper/2106.11796/) (2021)
- [Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model](https://mlsanity.com/paper/2210.17418/) (2022)
- [GKS: Graph-based Knowledge Selector for Task-oriented Dialog System](https://mlsanity.com/paper/2112.03719/) (2021)
- [Towards Generalized Models for Task-oriented Dialogue Modeling on Spoken Conversations](https://mlsanity.com/paper/2203.04045/) (2022)
- [Robust Unstructured Knowledge Access in Conversational Dialogue with ASR Errors](https://mlsanity.com/paper/2211.03990/) (2022)
- [Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems](https://mlsanity.com/paper/2212.09252/) (2022)
- [Retrieval-Augmented Neural Response Generation Using Logical Reasoning and Relevance Scoring](https://mlsanity.com/paper/2310.13566/) (2023)
- [HiNS: Hierarchical Negative Sampling for More Comprehensive Memory Retrieval Embedding Model](https://mlsanity.com/paper/2601.14857/) (2026)
- [External Knowledge Selection with Weighted Negative Sampling in Knowledge-grounded Task-oriented Dialogue Systems](https://mlsanity.com/paper/2209.02251/) (2022)
- [A Graph-to-Text Approach to Knowledge-Grounded Response Generation in Human-Robot Interaction](https://mlsanity.com/paper/2311.16137/) (2023)

## Abstract (arXiv)

The Track-1 of DSTC9 aims to effectively answer user requests or questions during task-oriented dialogues, which are out of the scope of APIs/DB. By leveraging external knowledge resources, relevant information can be retrieved and encoded into the response generation for these out-of-API-coverage queries. In this work, we have explored several advanced techniques to enhance the utilization of external knowledge and boost the quality of response generation, including schema guided knowledge decision, negatives enhanced knowledge selection, and knowledge grounded response generation. To evaluate the performance of our proposed method, comprehensive experiments have been carried out on the publicly available dataset. Our approach was ranked as the best in human evaluation of DSTC9 Track-1.