# Exploring the Advantages of Dense-Vector to One-Hot Encoding of Intent Classes in Out-of-Scope Detection Tasks

arXiv: https://arxiv.org/abs/2205.09021 · HTML: https://mlsanity.com/paper/2205.09021/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 18 May 2022.

## Builds on (most cited first, 23 of 23)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples](https://mlsanity.com/paper/1711.09325/) (2017)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Learning Confidence for Out-of-Distribution Detection in Neural Networks](https://mlsanity.com/paper/1802.04865/) (2018)
- [Zero-Shot Learning Through Cross-Modal Transfer](https://mlsanity.com/paper/1301.3666/) (2013)
- [Evaluation of Output Embeddings for Fine-Grained Image Classification](https://mlsanity.com/paper/1409.8403/) (2014)
- [Label-Embedding for Image Classification](https://mlsanity.com/paper/1503.08677/) (2015)
- [Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy](https://mlsanity.com/paper/1908.04951/) (2019)
- [Robust Out-of-distribution Detection for Neural Networks](https://mlsanity.com/paper/2003.09711/) (2020)
- [Out-of-distribution Detection in Classifiers via Generation](https://mlsanity.com/paper/1910.04241/) (2019)
- [Beyond One-hot Encoding: lower dimensional target embedding](https://mlsanity.com/paper/1806.10805/) (2018)
- [Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning](https://mlsanity.com/paper/2105.14289/) (2021)
- [Out-of-Domain Detection for Low-Resource Text Classification Tasks](https://mlsanity.com/paper/1909.05357/) (2019)
- [Out-of-Distribution Detection using Multiple Semantic Label Representations](https://mlsanity.com/paper/1808.06664/) (2018)
- [HINT3: Raising the bar for Intent Detection in the Wild](https://mlsanity.com/paper/2009.13833/) (2020)
- [Neural Sentence Embedding using Only In-domain Sentences for Out-of-domain Sentence Detection in Dialog Systems](https://mlsanity.com/paper/1807.11567/) (2018)
- [Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity](https://mlsanity.com/paper/1905.10628/) (2019)
- [kFolden: k-Fold Ensemble for Out-Of-Distribution Detection](https://mlsanity.com/paper/2108.12731/) (2021)

## Built on by (most cited first, 1 of 1)

- [Research on Joint Representation Learning Methods for Entity Neighborhood Information and Description Information](https://mlsanity.com/paper/2309.08100/) (2023)

## Abstract (arXiv)

This work explores the intrinsic limitations of the popular one-hot encoding method in classification of intents when detection of out-of-scope (OOS) inputs is required. Although recent work has shown that there can be significant improvements in OOS detection when the intent classes are represented as dense-vectors based on domain specific knowledge, we argue in this paper that such gains are more likely due to advantages of dense-vector to one-hot encoding methods in representing the complexity of the OOS space. We start by showing how dense-vector encodings can create OOS spaces with much richer topologies than one-hot encoding methods. We then demonstrate empirically, using four standard intent classification datasets, that knowledge-free, randomly generated dense-vector encodings of intent classes can yield massive, over 20% gains over one-hot encodings, and also outperform the previous, domain knowledge-based, SOTA of one of the datasets. We finish by describing a novel algorithm to search for good dense-vector encodings and present initial but promising experimental results of its use.