Disentangling Confidence Score Distribution for Out-of-Domain Intent Detection with Energy-Based Learning
Not on the map. Why?
Abstract (arXiv)
Detecting Out-of-Domain (OOD) or unknown intents from user queries is essential in a task-oriented dialog system. Traditional softmax-based confidence scores are susceptible to the overconfidence issue. In this paper, we propose a simple but strong energy-based score function to detect OOD where the energy scores of OOD samples are higher than IND samples. Further, given a small set of labeled OOD samples, we introduce an energy-based margin objective for supervised OOD detection to explicitly distinguish OOD samples from INDs. Comprehensive experiments and analysis prove our method helps disentangle confidence score distributions of IND and OOD data.footnoteOur code is available at urlhttps://github.com/pris-nlp/EMNLP2022-energyforOOD/.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 17 Oct 2022.
Builds on
20 of 32 references.
References · 20
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
- 2020Energy-based Out-of-distribution Detectioncited by 1,299
- 2015Towards Open Set Deep Networkscited by 538
- 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 453
- 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 405
- 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
- 2016Deep Structured Energy Based Models for Anomaly Detectioncited by 114
- 2017DOC: Deep Open Classification of Text Documentscited by 109
8 more
- 2018From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbotscited by 85
- 2019Deep Unknown Intent Detection with Margin Losscited by 66
- 2019Out-of-domain Detection for Natural Language Understanding in Dialog Systemscited by 62
- 2021Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learningcited by 42
- 2018Joint Learning of Domain Classification and Out-of-Domain Detection with Dynamic Class Weighting for Satisficing False Acceptance Ratescited by 17
- 2021Are all outliers alike? On Understanding the Diversity of Outliers for Detecting OODscited by 9
- 2017Chat Detection in an Intelligent Assistant: Combining Task-oriented and Non-task-oriented Spoken Dialogue Systemscited by 8
- 2022Distribution Calibration for Out-of-Domain Detection with Bayesian Approximationcited by 7
Built on by
Citing papers per year: 2023: 2.
2 papers.