Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection
Not on the map. Why?
Abstract (arXiv)
Detecting out-of-distribution (OOD) examples is critical in many applications. We propose an unsupervised method to detect OOD samples using a k-NN density estimate with respect to a classification model's intermediate activations on in-distribution samples. We leverage a recent insight about label smoothing, which we call the Label Smoothed Embedding Hypothesis, and show that one of the implications is that the k-NN density estimator performs better as an OOD detection method both theoretically and empirically when the model is trained with label smoothing. Finally, we show that our proposal outperforms many OOD baselines and also provide new finite-sample high-probability statistical results for k-NN density estimation's ability to detect OOD examples.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 60 references that are in the map. First posted on arXiv 9 Feb 2021.
Builds on
24 of 60 references.
References · 24
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
- 2019When Does Label Smoothing Help?cited by 985
- 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
- 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
- 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
- 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
- 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
- 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 211
12 more
- 2018To Trust Or Not To Trust A Classifiercited by 205
- 2020Does label smoothing mitigate label noise?cited by 204
- 2018Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learningcited by 203
- 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
- 2019PAC Confidence Sets for Deep Neural Networks via Calibrated Predictioncited by 59
- 2020Deep k-NN for Noisy Labelscited by 55
- 2017Efficient Defenses Against Adversarial Attackscited by 53
- 2016DisturbLabel: Regularizing CNN on the Loss Layercited by 41
- 2020Generalized Entropy Regularization or: There's Nothing Special about Label Smoothingcited by 40
- 2020Analysis of KNN Density Estimationcited by 28
- 2020Semantic Label Smoothing for Sequence to Sequence Problemscited by 13
- 2020An Investigation of how Label Smoothing Affects Generalizationcited by 11
Built on by
Citing papers per year: 2024: 1.
1 paper.