# Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System

arXiv: https://arxiv.org/abs/2105.14313 · HTML: https://mlsanity.com/paper/2105.14313/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 29 May 2021.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [DOC: Deep Open Classification of Text Documents](https://mlsanity.com/paper/1709.08716/) (2017)
- [A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1907.00390/) (2019)
- [Deep Unknown Intent Detection with Margin Loss](https://mlsanity.com/paper/1906.00434/) (2019)
- [Out-of-domain Detection for Natural Language Understanding in Dialog Systems](https://mlsanity.com/paper/1909.03862/) (2019)
- [Recent Neural Methods on Slot Filling and Intent Classification for Task-Oriented Dialogue Systems: A Survey](https://mlsanity.com/paper/2011.00564/) (2020)
- [Robust Zero-Shot Cross-Domain Slot Filling with Example Values](https://mlsanity.com/paper/1906.06870/) (2019)
- [Leveraging Knowledge Bases in LSTMs for Improving Machine Reading](https://mlsanity.com/paper/1902.09091/) (2019)
- [Few-Shot Representation Learning for Out-Of-Vocabulary Words](https://mlsanity.com/paper/1907.00505/) (2019)
- [Transfer Learning for Sequence Labeling Using Source Model and Target Data](https://mlsanity.com/paper/1902.05309/) (2019)

## Built on by (most cited first, 2 of 2)

- [CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog Evaluation](https://mlsanity.com/paper/2211.11617/) (2022)
- [Actively Discovering New Slots for Task-oriented Conversation](https://mlsanity.com/paper/2305.04049/) (2023)

## Abstract (arXiv)

Existing slot filling models can only recognize pre-defined in-domain slot types from a limited slot set. In the practical application, a reliable dialogue system should know what it does not know. In this paper, we introduce a new task, Novel Slot Detection (NSD), in the task-oriented dialogue system. NSD aims to discover unknown or out-of-domain slot types to strengthen the capability of a dialogue system based on in-domain training data. Besides, we construct two public NSD datasets, propose several strong NSD baselines, and establish a benchmark for future work. Finally, we conduct exhaustive experiments and qualitative analysis to comprehend key challenges and provide new guidance for future directions.