mlsanityData to 14 Oct 2026

When and How Does In-Distribution Label Help Out-of-Distribution Detection?

arXiv 2405.18635 ↗First posted 28 May 2024cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Detecting data points deviating from the training distribution is pivotal for ensuring reliable machine learning. Extensive research has been dedicated to the challenge, spanning classical anomaly detection techniques to contemporary out-of-distribution (OOD) detection approaches. While OOD detection commonly relies on supervised learning from a labeled in-distribution (ID) dataset, anomaly detection may treat the entire ID data as a single class and disregard ID labels. This fundamental distinction raises a significant question that has yet to be rigorously explored: when and how does ID label help OOD detection? This paper bridges this gap by offering a formal understanding to theoretically delineate the impact of ID labels on OOD detection. We employ a graph-theoretic approach, rigorously analyzing the separability of ID data from OOD data in a closed-form manner. Key to our approach is the characterization of data representations through spectral decomposition on the graph. Leveraging these representations, we establish a provable error bound that compares the OOD detection performance with and without ID labels, unveiling conditions for achieving enhanced OOD detection. Lastly, we present empirical results on both simulated and real datasets, validating theoretical guarantees and reinforcing our insights. Code is publicly available at https://github.com/deeplearning-wisc/id_label.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 71 of its 105 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

71 of 105 references.

References · 71

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  5. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  7. 2013Describing Textures in the Wildcited by 2,125
  8. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  9. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  10. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  11. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  12. 2021Generalized Out-of-Distribution Detection: A Surveycited by 802
38 more
  1. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
  2. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  3. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  4. 2015Towards Open Set Deep Networkscited by 538
  5. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  6. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  7. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  8. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
  9. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  10. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 339
  11. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  12. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 313
  13. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  14. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  15. 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
  16. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 252
  17. 2022ADBench: Anomaly Detection Benchmarkcited by 232
  18. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 231
  19. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 216
  20. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  21. 2021A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detectioncited by 207
  22. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 197
  23. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 179
  24. 2023Non-Parametric Outlier Synthesiscited by 129
  25. 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
  26. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  27. 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
  28. 2021Semantically Coherent Out-of-Distribution Detectioncited by 107
  29. 2019Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustnesscited by 104
  30. 2021Can multi-label classification networks know what they don't know?cited by 100
  31. 2018SpectralNet: Spectral Clustering using Deep Neural Networkscited by 94
  32. 2022Training OOD Detectors in their Natural Habitatscited by 86
  33. 2021Provable Guarantees for Understanding Out-of-distribution Detectioncited by 78
  34. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  35. 2022Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wildcited by 70
  36. 2019Towards neural networks that provably know when they don't knowcited by 69
  37. 2021On the Impact of Spurious Correlation for Out-of-distribution Detectioncited by 68
  38. 2023Energy-based Out-of-Distribution Detection for Graph Neural Networkscited by 68

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 1; 2026: 1.

6 papers.

  1. 2024Process Reward Model with Q-Value Rankingscited by 80
  2. 2026USBD: Universal Structural Basis Distillation for Source-Free Graph Domain Adaptationcited by 9
  3. 2024Rethinking Out-of-Distribution Detection on Imbalanced Data Distributioncited by 6
  4. 2024Out-of-Distribution Learning with Human Feedbackcited by 6
  5. 2025Can We Ignore Labels In Out of Distribution Detection?cited by 3
  6. 2024Semantic or Covariate? A Study on the Intractable Case of Out-of-Distribution Detectioncited by 0