# Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation

arXiv: https://arxiv.org/abs/2209.06612 · HTML: https://mlsanity.com/paper/2209.06612/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 23 references that are in the map.
- First posted on arXiv 14 Sep 2022.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots](https://mlsanity.com/paper/1801.01957/) (2018)
- [Deep Unknown Intent Detection with Margin Loss](https://mlsanity.com/paper/1906.00434/) (2019)
- [Out-of-domain Detection for Natural Language Understanding in Dialog Systems](https://mlsanity.com/paper/1909.03862/) (2019)
- [Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning](https://mlsanity.com/paper/2105.14289/) (2021)
- [Chat Detection in an Intelligent Assistant: Combining Task-oriented and Non-task-oriented Spoken Dialogue Systems](https://mlsanity.com/paper/1705.00746/) (2017)

## Built on by (most cited first, 7 of 7)

- [Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection](https://mlsanity.com/paper/2402.17256/) (2024)
- [Continual Generalized Intent Discovery: Marching Towards Dynamic and Open-world Intent Recognition](https://mlsanity.com/paper/2310.10184/) (2023)
- [Can LLMs Detect Their Confabulations? Estimating Reliability in Uncertainty-Aware Language Models](https://mlsanity.com/paper/2508.08139/) (2025)
- [SELFOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank](https://mlsanity.com/paper/2305.14696/) (2023)
- [APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection](https://mlsanity.com/paper/2310.13380/) (2023)
- [Disentangling Confidence Score Distribution for Out-of-Domain Intent Detection with Energy-Based Learning](https://mlsanity.com/paper/2210.08830/) (2022)
- [Rethinking and Recomputing the Value of Machine Learning Models](https://mlsanity.com/paper/2209.15157/) (2022)

## Abstract (arXiv)

Out-of-Domain (OOD) detection is a key component in a task-oriented dialog system, which aims to identify whether a query falls outside the predefined supported intent set. Previous softmax-based detection algorithms are proved to be overconfident for OOD samples. In this paper, we analyze overconfident OOD comes from distribution uncertainty due to the mismatch between the training and test distributions, which makes the model can't confidently make predictions thus probably causing abnormal softmax scores. We propose a Bayesian OOD detection framework to calibrate distribution uncertainty using Monte-Carlo Dropout. Our method is flexible and easily pluggable into existing softmax-based baselines and gains 33.33% OOD F1 improvements with increasing only 0.41% inference time compared to MSP. Further analyses show the effectiveness of Bayesian learning for OOD detection.