# Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection

arXiv: https://arxiv.org/abs/2101.03778 · HTML: https://mlsanity.com/paper/2101.03778/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 11 Jan 2021.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog](https://mlsanity.com/paper/1810.13327/) (2018)
- [Deep Unknown Intent Detection with Margin Loss](https://mlsanity.com/paper/1906.00434/) (2019)
- [Out-of-domain Detection for Natural Language Understanding in Dialog Systems](https://mlsanity.com/paper/1909.03862/) (2019)
- [Out-of-Domain Detection for Low-Resource Text Classification Tasks](https://mlsanity.com/paper/1909.05357/) (2019)
- [Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog](https://mlsanity.com/paper/1912.12800/) (2019)
- [Distance-based Confidence Score for Neural Network Classifiers](https://mlsanity.com/paper/1709.09844/) (2017)
- [Statistical Analysis of Nearest Neighbor Methods for Anomaly Detection](https://mlsanity.com/paper/1907.03813/) (2019)
- [Why is the Mahalanobis Distance Effective for Anomaly Detection?](https://mlsanity.com/paper/2003.00402/) (2020)
- [Robustness properties of Facebook's ResNeXt WSL models](https://mlsanity.com/paper/1907.07640/) (2019)

## Built on by (most cited first, 30 of 70)

- [Delving into Out-of-Distribution Detection with Vision-Language Representations](https://mlsanity.com/paper/2211.13445/) (2022)
- [POEM: Out-of-Distribution Detection with Posterior Sampling](https://mlsanity.com/paper/2206.13687/) (2022)
- [Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph](https://mlsanity.com/paper/2406.15627/) (2024)
- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [GL-MCM: Global and Local Maximum Concept Matching for Zero-Shot Out-of-Distribution Detection](https://mlsanity.com/paper/2304.04521/) (2023)
- [A Survey on Out-of-Distribution Detection in NLP](https://mlsanity.com/paper/2305.03236/) (2023)
- [Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detection](https://mlsanity.com/paper/2305.13282/) (2023)
- [How is BERT surprised? Layerwise detection of linguistic anomalies](https://mlsanity.com/paper/2105.07452/) (2021)
- [Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data](https://mlsanity.com/paper/2212.09171/) (2022)
- [BadActs: A Universal Backdoor Defense in the Activation Space](https://mlsanity.com/paper/2405.11227/) (2024)
- [GOLD: Improving Out-of-Scope Detection in Dialogues using Data Augmentation](https://mlsanity.com/paper/2109.03079/) (2021)
- [Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks](https://mlsanity.com/paper/2210.07907/) (2022)
- [Robust AI-Generated Text Detection by Restricted Embeddings](https://mlsanity.com/paper/2410.08113/) (2024)
- [Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models](https://mlsanity.com/paper/2502.14427/) (2025)
- [How Good Are LLMs at Out-of-Distribution Detection?](https://mlsanity.com/paper/2308.10261/) (2023)
- [Unsupervised Layer-wise Score Aggregation for Textual OOD Detection](https://mlsanity.com/paper/2302.09852/) (2023)
- [Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness](https://mlsanity.com/paper/2310.02832/) (2023)
- [Towards Textual Out-of-Domain Detection without In-Domain Labels](https://mlsanity.com/paper/2203.11396/) (2022)
- [Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features](https://mlsanity.com/paper/2301.12715/) (2023)
- [Towards Computationally Feasible Deep Active Learning](https://mlsanity.com/paper/2205.03598/) (2022)
- [Estimating Soft Labels for Out-of-Domain Intent Detection](https://mlsanity.com/paper/2211.05561/) (2022)
- [Multi-Level Knowledge Distillation for Out-of-Distribution Detection in Text](https://mlsanity.com/paper/2211.11300/) (2022)
- [Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector](https://mlsanity.com/paper/2404.08679/) (2024)
- [Uncertainty Estimation of Transformers' Predictions via Topological Analysis of the Attention Matrices](https://mlsanity.com/paper/2308.11295/) (2023)
- [Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation](https://mlsanity.com/paper/2205.13346/) (2022)
- [Shh, don't say that! Domain Certification in LLMs](https://mlsanity.com/paper/2502.19320/) (2025)
- [Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection](https://mlsanity.com/paper/2510.05782/) (2025)
- [Holistic Sentence Embeddings for Better Out-of-Distribution Detection](https://mlsanity.com/paper/2210.07485/) (2022)
- [Online Reinforcement Learning in Non-Stationary Context-Driven Environments](https://mlsanity.com/paper/2302.02182/) (2023)
- [Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contexts](https://mlsanity.com/paper/2305.03237/) (2023)

## Abstract (arXiv)

Real-life applications, heavily relying on machine learning, such as dialog systems, demand out-of-domain detection methods. Intent classification models should be equipped with a mechanism to distinguish seen intents from unseen ones so that the dialog agent is capable of rejecting the latter and avoiding undesired behavior. However, despite increasing attention paid to the task, the best practices for out-of-domain intent detection have not yet been fully established. This paper conducts a thorough comparison of out-of-domain intent detection methods. We prioritize the methods, not requiring access to out-of-domain data during training, gathering of which is extremely time- and labor-consuming due to lexical and stylistic variation of user utterances. We evaluate multiple contextual encoders and methods, proven to be efficient, on three standard datasets for intent classification, expanded with out-of-domain utterances. Our main findings show that fine-tuning Transformer-based encoders on in-domain data leads to superior results. Mahalanobis distance, together with utterance representations, derived from Transformer-based encoders, outperforms other methods by a wide margin and establishes new state-of-the-art results for all datasets. The broader analysis shows that the reason for success lies in the fact that the fine-tuned Transformer is capable of constructing homogeneous representations of in-domain utterances, revealing geometrical disparity to out of domain utterances. In turn, the Mahalanobis distance captures this disparity easily. The code is available in our GitHub repo: https://github.com/huawei-noah/noah-research/tree/master/Maha_OOD .