# Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLU

arXiv: https://arxiv.org/abs/2106.14464 · HTML: https://mlsanity.com/paper/2106.14464/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [Out-of-domain Detection for Natural Language Understanding in Dialog Systems](https://mlsanity.com/paper/1909.03862/) (2019)
- [A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling](https://mlsanity.com/paper/1812.10235/) (2018)
- [Out-of-Domain Detection for Low-Resource Text Classification Tasks](https://mlsanity.com/paper/1909.05357/) (2019)
- [Neural Sentence Embedding using Only In-domain Sentences for Out-of-domain Sentence Detection in Dialog Systems](https://mlsanity.com/paper/1807.11567/) (2018)
- [OneNet: Joint Domain, Intent, Slot Prediction for Spoken Language Understanding](https://mlsanity.com/paper/1801.05149/) (2018)

## Built on by (most cited first, 22 of 22)

- [Delving into Out-of-Distribution Detection with Vision-Language Representations](https://mlsanity.com/paper/2211.13445/) (2022)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective](https://mlsanity.com/paper/2211.08073/) (2022)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [A Survey on Out-of-Distribution Detection in NLP](https://mlsanity.com/paper/2305.03236/) (2023)
- [Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detection](https://mlsanity.com/paper/2305.13282/) (2023)
- [Learning Discriminative Representations and Decision Boundaries for Open Intent Detection](https://mlsanity.com/paper/2203.05823/) (2022)
- [Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection](https://mlsanity.com/paper/2310.08027/) (2023)
- [Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features](https://mlsanity.com/paper/2301.12715/) (2023)
- [Estimating Soft Labels for Out-of-Domain Intent Detection](https://mlsanity.com/paper/2211.05561/) (2022)
- [Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble](https://mlsanity.com/paper/2210.11034/) (2022)
- [Paraphrase and Aggregate with Large Language Models for Minimizing Intent Classification Errors](https://mlsanity.com/paper/2406.17163/) (2024)
- [A Simple Meta-learning Paradigm for Zero-shot Intent Classification with Mixture Attention Mechanism](https://mlsanity.com/paper/2206.02179/) (2022)
- [Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contexts](https://mlsanity.com/paper/2305.03237/) (2023)
- [SimLabel: Consistency-Guided OOD Detection with Pretrained Vision-Language Models](https://mlsanity.com/paper/2501.11485/) (2025)
- [Distinguish Sense from Nonsense: Out-of-Scope Detection for Virtual Assistants](https://mlsanity.com/paper/2301.06544/) (2023)
- [Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysis](https://mlsanity.com/paper/2402.14536/) (2024)
- [Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2301.11660/) (2023)
- [Improving End-to-End Speech Processing by Efficient Text Data Utilization with Latent Synthesis](https://mlsanity.com/paper/2310.05374/) (2023)
- [One Stone, Four Birds: A Comprehensive Solution for QA System Using Supervised Contrastive Learning](https://mlsanity.com/paper/2407.09011/) (2024)
- [From Prediction to Justification: Aligning Sentiment Reasoning with Human Rationale via Reinforcement Learning](https://mlsanity.com/paper/2604.13398/) (2026)

## Abstract (arXiv)

Intent classification is a major task in spoken language understanding (SLU). Since most models are built with pre-collected in-domain (IND) training utterances, their ability to detect unsupported out-of-domain (OOD) utterances has a critical effect in practical use. Recent works have shown that using extra data and labels can improve the OOD detection performance, yet it could be costly to collect such data. This paper proposes to train a model with only IND data while supporting both IND intent classification and OOD detection. Our method designs a novel domain-regularized module (DRM) to reduce the overconfident phenomenon of a vanilla classifier, achieving a better generalization in both cases. Besides, DRM can be used as a drop-in replacement for the last layer in any neural network-based intent classifier, providing a low-cost strategy for a significant improvement. The evaluation on four datasets shows that our method built on BERT and RoBERTa models achieves state-of-the-art performance against existing approaches and the strong baselines we created for the comparisons.