mlsanityData to 7 Oct 2026

Leveraging Intermediate Representations for Better Out-of-Distribution Detection

arXiv 2502.12849 ↗First posted 18 Feb 2025cs.LG cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

In real-world applications, machine learning models must reliably detect Out-of-Distribution (OoD) samples to prevent unsafe decisions. Current OoD detection methods often rely on analyzing the logits or the embeddings of the penultimate layer of a neural network. However, little work has been conducted on the exploitation of the rich information encoded in intermediate layers. To address this, we analyze the discriminative power of intermediate layers and show that they can positively be used for OoD detection. Therefore, we propose to regularize intermediate layers with an energy-based contrastive loss, and by grouping multiple layers in a single aggregated response. We demonstrate that intermediate layer activations improves OoD detection performance by running a comprehensive evaluation across multiple datasets.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 35 of its 44 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

35 of 44 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2013Describing Textures in the Wildcited by 2,110
  8. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  9. 2016Concrete Problems in AI Safetycited by 1,582
  10. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  11. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  12. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
23 more
  1. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  2. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  3. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  4. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  5. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  6. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  7. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
  8. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  9. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  10. 2018Learning Confidence for Out-of-Distribution Detection in Neural Networkscited by 245
  11. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  12. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 178
  13. 2023In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluationcited by 132
  14. 2022Head2Toe: Utilizing Intermediate Representations for Better Transfer Learningcited by 83
  15. 2023Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulationcited by 60
  16. 2023Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolationcited by 38
  17. 2023Learning to Augment Distributions for Out-of-Distribution Detectioncited by 38
  18. 2017The iNaturalist Species Classification and Detection Datasetcited by 28
  19. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 24
  20. 2018Metric Learning for Novelty and Anomaly Detectioncited by 21
  21. 2024Dissecting Out-of-Distribution Detection and Open-Set Recognition: A Critical Analysis of Methods and Benchmarkscited by 13
  22. 2023Gaussian Latent Representations for Uncertainty Estimation using Mahalanobis Distance in Deep Classifierscited by 10
  23. 2021Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Datacited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2025Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detectioncited by 6
  2. 2026Prototype Fusion: A Training-Free Multi-Layer Approach to OOD Detectioncited by 0
  3. 2026Ranked Activation Shift for Post-Hoc Out-of-Distribution Detectioncited by 0
  4. 2026Sparse Autoencoders for Interpretable Out-of-Distribution Detectioncited by 0
  5. 2026Not All Layers Are Equal: Dynamic Layer Routing for Reliable CLIP OOD Detectioncited by 0