# Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regression

arXiv: https://arxiv.org/abs/2407.13141 · HTML: https://mlsanity.com/paper/2407.13141/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 25 of 25)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Out-of-Distribution Detection with Deep Nearest Neighbors](https://mlsanity.com/paper/2204.06507/) (2022)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets](https://mlsanity.com/paper/2008.02637/) (2020)
- [Contrastive Out-of-Distribution Detection for Pretrained Transformers](https://mlsanity.com/paper/2104.08812/) (2021)
- [Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection](https://mlsanity.com/paper/2101.03778/) (2021)
- [Deep Open Intent Classification with Adaptive Decision Boundary](https://mlsanity.com/paper/2012.10209/) (2020)
- [Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training](https://mlsanity.com/paper/2106.08616/) (2021)
- [What to do about non-standard (or non-canonical) language in NLP](https://mlsanity.com/paper/1608.07836/) (2016)
- [Unsupervised Out-of-Domain Detection via Pre-trained Transformers](https://mlsanity.com/paper/2106.00948/) (2021)
- [DATE: Detecting Anomalies in Text via Self-Supervision of Transformers](https://mlsanity.com/paper/2104.05591/) (2021)
- [Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness](https://mlsanity.com/paper/2310.02832/) (2023)
- [Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features](https://mlsanity.com/paper/2301.12715/) (2023)
- [Estimating Soft Labels for Out-of-Domain Intent Detection](https://mlsanity.com/paper/2211.05561/) (2022)
- [Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained Transformers](https://mlsanity.com/paper/2307.09455/) (2023)
- [FLatS: Principled Out-of-Distribution Detection with Feature-Based Likelihood Ratio Score](https://mlsanity.com/paper/2310.05083/) (2023)
- [NNK-Means: Data summarization using dictionary learning with non-negative kernel regression](https://mlsanity.com/paper/2110.08212/) (2021)

## Built on by (most cited first, 3 of 3)

- [Distribution Shifts at Scale: Out-of-distribution Detection in Earth Observation](https://mlsanity.com/paper/2412.13394/) (2024)
- [Efficient Out-of-Scope Detection in Dialogue Systems via Uncertainty-Driven LLM Routing](https://mlsanity.com/paper/2507.01541/) (2025)
- [Scaling Pretrained Representations Enables Label-Free Out-of-Distribution Detection Without Fine-Tuning](https://mlsanity.com/paper/2605.05638/) (2026)

## Abstract (arXiv)

As language models become more general purpose, increased attention needs to be paid to detecting out-of-distribution (OOD) instances, i.e., those not belonging to any of the distributions seen during training. Existing methods for detecting OOD data are computationally complex and storage-intensive. We propose a novel soft clustering approach for OOD detection based on non-negative kernel regression. Our approach greatly reduces computational and space complexities (up to 11x improvement in inference time and 87% reduction in storage requirements) and outperforms existing approaches by up to 4 AUROC points on four different benchmarks. We also introduce an entropy-constrained version of our algorithm, which leads to further reductions in storage requirements (up to 97% lower than comparable approaches) while retaining competitive performance. Our soft clustering approach for OOD detection highlights its potential for detecting tail-end phenomena in extreme-scale data settings.