mlsanityData to 14 Oct 2026

An Information-Theoretic Framework for Feature Construction in Out-of-Distribution Detection

arXiv 2506.14194 ↗First posted 17 Jun 2025cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We present a theory for the construction of out-of-distribution (OOD) detection features for neural networks. We introduce random features for OOD through a novel information-theoretic loss functional consisting of two terms, the first based on the KL divergence separates resulting in-distribution (ID) and OOD feature distributions and the second term is the Information Bottleneck, which favors compressed features that retain the OOD information. We formulate a variational procedure to optimize the loss and obtain OOD features. Based on assumptions on OOD distributions, one can recover properties of existing OOD features, i.e., shaping functions. Furthermore, we show that our theory can predict a new shaping function that out-performs existing ones on OOD benchmarks, including both semantic and a wide range of covariate shifts. Our theory provides a general framework for constructing a variety of new features with clear explainability.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 39 of its 57 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

39 of 57 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  10. 2013Describing Textures in the Wildcited by 2,125
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  12. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
27 more
  1. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  2. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  3. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  4. 2019Natural Adversarial Examplescited by 1,408
  5. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  6. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  7. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  8. 2018A Tutorial on Bayesian Optimizationcited by 744
  9. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
  10. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  11. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  12. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 453
  13. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 405
  14. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 353
  15. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  16. 2018Learning Confidence for Out-of-Distribution Detection in Neural Networkscited by 245
  17. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 221
  18. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  19. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 179
  20. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  21. 2015TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Trackingcited by 165
  22. 2023Energy-based Out-of-Distribution Detection for Graph Neural Networkscited by 68
  23. 2022Full-Spectrum Out-of-Distribution Detectioncited by 63
  24. 2023ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithmscited by 22
  25. 2021On Out-of-distribution Detection with Energy-based Modelscited by 17
  26. 2024Towards Optimal Feature-Shaping Methods for Out-of-Distribution Detectioncited by 13
  27. 2023Expecting The Unexpected: Towards Broad Out-Of-Distribution Detectioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Information-Theoretic Reward Modeling for Stable RLHF: Detecting and Mitigating Reward Hackingcited by 5
  2. 2026Tuning Out-of-Distribution (OOD) Detectors Without Given OOD Datacited by 2
  3. 2025Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Directioncited by 0
  4. 2026Out-of-Distribution (OOD) Detectors for Open-Set RF Fingerprintingcited by 0