# Multi-layer Aggregation as a key to feature-based OOD detection

arXiv: https://arxiv.org/abs/2307.15647 · HTML: https://mlsanity.com/paper/2307.15647/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 28 Jul 2023.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge](https://mlsanity.com/paper/1904.00682/) (2019)
- [On the Practicality of Deterministic Epistemic Uncertainty](https://mlsanity.com/paper/2107.00649/) (2021)
- [Confidence-based Out-of-Distribution Detection: A Comparative Study and Analysis](https://mlsanity.com/paper/2107.02568/) (2021)
- [Automatic quality control of brain T1-weighted magnetic resonance images for a clinical data warehouse](https://mlsanity.com/paper/2104.08131/) (2021)
- [Simulation of Brain Resection for Cavity Segmentation Using Self-Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/2006.15693/) (2020)
- [Layer Adaptive Deep Neural Networks for Out-of-distribution Detection](https://mlsanity.com/paper/2203.00192/) (2022)

## Built on by (most cited first, 5 of 5)

- [Towards Integrating Epistemic Uncertainty Estimation into the Radiotherapy Workflow](https://mlsanity.com/paper/2409.18628/) (2024)
- [Safeguarding AI in Medical Imaging: Post-Hoc Out-of-Distribution Detection with Normalizing Flows](https://mlsanity.com/paper/2502.11638/) (2025)
- [The Invisible Gorilla Effect in Out-of-distribution Detection](https://mlsanity.com/paper/2602.20068/) (2026)
- [Prototype Fusion: A Training-Free Multi-Layer Approach to OOD Detection](https://mlsanity.com/paper/2603.23677/) (2026)
- [Layer Selection in VLMs for Zero-Shot OOD Detection via Multi-Resolution Entropy Estimation](https://mlsanity.com/paper/2609.08524/) (2026)

## Abstract (arXiv)

Deep Learning models are easily disturbed by variations in the input images that were not observed during the training stage, resulting in unpredictable predictions. Detecting such Out-of-Distribution (OOD) images is particularly crucial in the context of medical image analysis, where the range of possible abnormalities is extremely wide. Recently, a new category of methods has emerged, based on the analysis of the intermediate features of a trained model. These methods can be divided into 2 groups: single-layer methods that consider the feature map obtained at a fixed, carefully chosen layer, and multi-layer methods that consider the ensemble of the feature maps generated by the model. While promising, a proper comparison of these algorithms is still lacking. In this work, we compared various feature-based OOD detection methods on a large spectra of OOD (20 types), representing approximately 7800 3D MRIs. Our experiments shed the light on two phenomenons. First, multi-layer methods consistently outperform single-layer approaches, which tend to have inconsistent behaviour depending on the type of anomaly. Second, the OOD detection performance highly depends on the architecture of the underlying neural network.