# Out-of-domain Detection for Natural Language Understanding in Dialog Systems

arXiv: https://arxiv.org/abs/1909.03862 · HTML: https://mlsanity.com/paper/1909.03862/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2019.

## Built on by (most cited first, 30 of 62)

- [Delving into Out-of-Distribution Detection with Vision-Language Representations](https://mlsanity.com/paper/2211.13445/) (2022)
- [LLM Internal States Reveal Hallucination Risk Faced With a Query](https://mlsanity.com/paper/2407.03282/) (2024)
- [Types of Out-of-Distribution Texts and How to Detect Them](https://mlsanity.com/paper/2109.06827/) (2021)
- [GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective](https://mlsanity.com/paper/2211.08073/) (2022)
- [Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection](https://mlsanity.com/paper/2101.03778/) (2021)
- [A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness](https://mlsanity.com/paper/2205.00403/) (2022)
- [EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Records](https://mlsanity.com/paper/2301.07695/) (2023)
- [Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning](https://mlsanity.com/paper/2105.14289/) (2021)
- [A Survey on Out-of-Distribution Detection in NLP](https://mlsanity.com/paper/2305.03236/) (2023)
- [Uncertainty-Aware Reliable Text Classification](https://mlsanity.com/paper/2107.07114/) (2021)
- [Are Pretrained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection](https://mlsanity.com/paper/2106.04564/) (2021)
- [Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLU](https://mlsanity.com/paper/2106.14464/) (2021)
- [GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentation](https://mlsanity.com/paper/2109.03079/) (2021)
- [On Language Models for Creoles](https://mlsanity.com/paper/2109.06074/) (2021)
- [TrustSQL: Benchmarking Text-to-SQL Reliability with Penalty-Based Scoring](https://mlsanity.com/paper/2403.15879/) (2024)
- [OodGAN: Generative Adversarial Network for Out-of-Domain Data Generation](https://mlsanity.com/paper/2104.02484/) (2021)
- [Learning Discriminative Representations and Decision Boundaries for Open Intent Detection](https://mlsanity.com/paper/2203.05823/) (2022)
- [CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues](https://mlsanity.com/paper/2404.03820/) (2024)
- [Towards Textual Out-of-Domain Detection without In-Domain Labels](https://mlsanity.com/paper/2203.11396/) (2022)
- [A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialog](https://mlsanity.com/paper/2207.13211/) (2022)
- [Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection](https://mlsanity.com/paper/2402.17256/) (2024)
- [Estimating Soft Labels for Out-of-Domain Intent Detection](https://mlsanity.com/paper/2211.05561/) (2022)
- [Generalized Intent Discovery: Learning from Open World Dialogue System](https://mlsanity.com/paper/2209.06030/) (2022)
- [SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation](https://mlsanity.com/paper/2406.12629/) (2024)
- [Causal Agent based on Large Language Model](https://mlsanity.com/paper/2408.06849/) (2024)
- [Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection](https://mlsanity.com/paper/2409.10985/) (2024)
- [Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble](https://mlsanity.com/paper/2210.11034/) (2022)
- [Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation](https://mlsanity.com/paper/2209.06612/) (2022)
- [Energy-based Unknown Intent Detection with Data Manipulation](https://mlsanity.com/paper/2107.12542/) (2021)
- [Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contexts](https://mlsanity.com/paper/2305.03237/) (2023)

## Abstract (arXiv)

Natural Language Understanding (NLU) is a vital component of dialogue systems, and its ability to detect Out-of-Domain (OOD) inputs is critical in practical applications, since the acceptance of the OOD input that is unsupported by the current system may lead to catastrophic failure. However, most existing OOD detection methods rely heavily on manually labeled OOD samples and cannot take full advantage of unlabeled data. This limits the feasibility of these models in practical applications. In this paper, we propose a novel model to generate high-quality pseudo OOD samples that are akin to IN-Domain (IND) input utterances, and thereby improves the performance of OOD detection. To this end, an autoencoder is trained to map an input utterance into a latent code. and the codes of IND and OOD samples are trained to be indistinguishable by utilizing a generative adversarial network. To provide more supervision signals, an auxiliary classifier is introduced to regularize the generated OOD samples to have indistinguishable intent labels. Experiments show that these pseudo OOD samples generated by our model can be used to effectively improve OOD detection in NLU. Besides, we also demonstrate that the effectiveness of these pseudo OOD data can be further improved by efficiently utilizing unlabeled data.