# Layer Adaptive Deep Neural Networks for Out-of-distribution Detection

arXiv: https://arxiv.org/abs/2203.00192 · HTML: https://mlsanity.com/paper/2203.00192/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 1 Mar 2022.

## Builds on (most cited first, 15 of 15)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking](https://mlsanity.com/paper/1504.06755/) (2015)
- [Adaptive Neural Networks for Efficient Inference](https://mlsanity.com/paper/1702.07811/) (2017)
- [Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifiers](https://mlsanity.com/paper/1809.03576/) (2018)
- [Interpreting Deep Visual Representations via Network Dissection](https://mlsanity.com/paper/1711.05611/) (2017)
- [Detecting Out-of-Distribution Inputs in Deep Neural Networks Using an Early-Layer Output](https://mlsanity.com/paper/1910.10307/) (2019)
- [Rank-Based Multi-task Learning for Fair Regression](https://mlsanity.com/paper/2009.11405/) (2020)
- [Multi-Class Data Description for Out-of-distribution Detection](https://mlsanity.com/paper/2104.00941/) (2021)
- [Fairness-Aware Online Meta-learning](https://mlsanity.com/paper/2108.09435/) (2021)
- [Multi-Scale Dense Networks for Resource Efficient Image Classification](https://mlsanity.com/paper/1703.09844/) (2017)

## Built on by (most cited first, 4 of 4)

- [Unsupervised Layer-wise Score Aggregation for Textual OOD Detection](https://mlsanity.com/paper/2302.09852/) (2023)
- [Multi-layer Aggregation as a key to feature-based OOD detection](https://mlsanity.com/paper/2307.15647/) (2023)
- [MADOD: Generalizing OOD Detection to Unseen Domains via G-Invariance Meta-Learning](https://mlsanity.com/paper/2411.02444/) (2024)
- [Feature-Space Semantic Invariance: Enhanced OOD Detection for Open-Set Domain Generalization](https://mlsanity.com/paper/2411.07392/) (2024)

## Abstract (arXiv)

During the forward pass of Deep Neural Networks (DNNs), inputs gradually transformed from low-level features to high-level conceptual labels. While features at different layers could summarize the important factors of the inputs at varying levels, modern out-of-distribution (OOD) detection methods mostly focus on utilizing their ending layer features. In this paper, we proposed a novel layer-adaptive OOD detection framework (LA-OOD) for DNNs that can fully utilize the intermediate layers' outputs. Specifically, instead of training a unified OOD detector at a fixed ending layer, we train multiple One-Class SVM OOD detectors simultaneously at the intermediate layers to exploit the full spectrum characteristics encoded at varying depths of DNNs. We develop a simple yet effective layer-adaptive policy to identify the best layer for detecting each potential OOD example. LA-OOD can be applied to any existing DNNs and does not require access to OOD samples during the training. Using three DNNs of varying depth and architectures, our experiments demonstrate that LA-OOD is robust against OODs of varying complexity and can outperform state-of-the-art competitors by a large margin on some real-world datasets.