# When Confidence Lacks Concepts: Interpretable OOD Detection via Representation Perturbations

arXiv: https://arxiv.org/abs/2606.16196 · HTML: https://mlsanity.com/paper/2606.16196/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 15 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [Scaling Out-of-Distribution Detection for Real-World Settings](https://mlsanity.com/paper/1911.11132/) (2019)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [ViM: Out-Of-Distribution with Virtual-logit Matching](https://mlsanity.com/paper/2203.10807/) (2022)
- [On the Importance of Gradients for Detecting Distributional Shifts in the Wild](https://mlsanity.com/paper/2110.00218/) (2021)
- [Deep Clustering: A Comprehensive Survey](https://mlsanity.com/paper/2210.04142/) (2022)
- [SAEs Are Good for Steering -- If You Select the Right Features](https://mlsanity.com/paper/2505.20063/) (2025)
- [Density of States Estimation for Out-of-Distribution Detection](https://mlsanity.com/paper/2006.09273/) (2020)
- [Anomaly Detection in Medical Imaging -- A Mini Review](https://mlsanity.com/paper/2108.11986/) (2021)
- [NECO: NEural Collapse Based Out-of-distribution detection](https://mlsanity.com/paper/2310.06823/) (2023)
- [Interpretable and Testable Vision Features via Sparse Autoencoders](https://mlsanity.com/paper/2502.06755/) (2025)
- [Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Video](https://mlsanity.com/paper/2504.19475/) (2025)
- [Concept-based Explanations for Out-Of-Distribution Detectors](https://mlsanity.com/paper/2203.02586/) (2022)
- [PathoDuet: Foundation Models for Pathological Slide Analysis of H&E and IHC Stains](https://mlsanity.com/paper/2312.09894/) (2023)
- [Handling Out-of-Distribution Data: A Survey](https://mlsanity.com/paper/2507.21160/) (2025)
- [Image-based Outlier Synthesis With Training Data](https://mlsanity.com/paper/2411.10794/) (2024)
- [NERO: Explainable Out-of-Distribution Detection with Neuron-level Relevance](https://mlsanity.com/paper/2506.15404/) (2025)

## Abstract (arXiv)

Deep neural networks have achieved remarkable performance across medical imaging tasks, yet their tendency to overgeneralize under distributional shifts poses a major obstacle to safe clinical deployment. Out-of-Distribution (OOD) detection methods aim to mitigate this risk, but most existing approaches rely on opaque internal signals with poorly understood semantic meaning, limiting trust in safety-critical settings. In this work, we propose an interpretable OOD detection framework that probes the stability of model predictions under class-conditioned semantic perturbations. Leveraging sparse autoencoders (SAEs), we learn class-specific concept vectors from in-distribution data that disentangle dense intermediate representations into sparse, semantically meaningful components. At inference, we perturb deeper-layer representations using the concept vectors associated with the model's predicted class and measure the class logits stability. We hypothesize that in-distribution samples exhibit low sensitivity to such perturbations, as their representations align with class-specific semantic directions, whereas OOD samples show amplified deviations due to representational misalignment. By framing OOD detection as a concept conditioned stability analysis, our approach provides both a discriminative OOD signal and an interpretable lens into the internal mechanisms driving model uncertainty, making it particularly suitable for high stakes medical applications.