# Intermediate Layer Classifiers for OOD generalization

arXiv: https://arxiv.org/abs/2504.05461 · HTML: https://mlsanity.com/paper/2504.05461/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 67 references that are in the map.
- First posted on arXiv 7 Apr 2025.

## Builds on (most cited first, 30 of 51)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark](https://mlsanity.com/paper/1910.04867/) (2019)
- [Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations](https://mlsanity.com/paper/2204.02937/) (2022)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [Gradient Starvation: A Learning Proclivity in Neural Networks](https://mlsanity.com/paper/2011.09468/) (2020)
- [Surgical Fine-Tuning Improves Adaptation to Distribution Shifts](https://mlsanity.com/paper/2210.11466/) (2022)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [Deep learning generalizes because the parameter-function map is biased towards simple functions](https://mlsanity.com/paper/1805.08522/) (2018)

## Built on by (most cited first, 14 of 14)

- [Does Data Scaling Lead to Visual Compositional Generalization?](https://mlsanity.com/paper/2507.07102/) (2025)
- [Diffusion Classifiers Understand Compositionality, but Conditions Apply](https://mlsanity.com/paper/2505.17955/) (2025)
- [A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning](https://mlsanity.com/paper/2601.06851/) (2026)
- [On the rankability of visual embeddings](https://mlsanity.com/paper/2507.03683/) (2025)
- [Deep Models, Shallow Alignment: Uncovering the Granularity Mismatch in Neural Decoding](https://mlsanity.com/paper/2601.21948/) (2026)
- [The Generalization Ridge: Information Flow in Natural Language Generation](https://mlsanity.com/paper/2507.05387/) (2025)
- [Rethinking the Use of Vision Transformers for AI-Generated Image Detection](https://mlsanity.com/paper/2512.04969/) (2025)
- [Layer by layer, module by module: Choose both for optimal OOD probing of ViT](https://mlsanity.com/paper/2603.05280/) (2026)
- [Fast Wireless Foundation Models with Early-Exits](https://mlsanity.com/paper/2606.29640/) (2026)
- [DIRCR: Dual-Inference Rule-Contrastive Reasoning for Solving RAVENs](https://mlsanity.com/paper/2604.17584/) (2026)
- [Vertical Fusion: Condensing Internal Representations for Robust ViT Classification](https://mlsanity.com/paper/2607.10391/) (2026)
- [Representation Trajectories Matters: Complementary Evidence for OOD Detection and Image Classification](https://mlsanity.com/paper/2607.26565/) (2026)
- [Not All Layers Are Equal: Dynamic Layer Routing for Reliable CLIP OOD Detection](https://mlsanity.com/paper/2609.20299/) (2026)
- [The Layer Mystery of VLA: An Information-Theoretical Analysis of VLA Latent Interface](https://mlsanity.com/paper/2609.36118/) (2026)

## Abstract (arXiv)

Deep classifiers are known to be sensitive to data distribution shifts, primarily due to their reliance on spurious correlations in training data. It has been suggested that these classifiers can still find useful features in the network's last layer that hold up under such shifts. In this work, we question the use of last-layer representations for out-of-distribution (OOD) generalisation and explore the utility of intermediate layers. To this end, we introduce textitIntermediate Layer Classifiers (ILCs). We discover that intermediate layer representations frequently offer substantially better generalisation than those from the penultimate layer. In many cases, zero-shot OOD generalisation using earlier-layer representations approaches the few-shot performance of retraining on penultimate layer representations. This is confirmed across multiple datasets, architectures, and types of distribution shifts. Our analysis suggests that intermediate layers are less sensitive to distribution shifts compared to the penultimate layer. These findings highlight the importance of understanding how information is distributed across network layers and its role in OOD generalisation, while also pointing to the limits of penultimate layer representation utility. Code is available at https://github.com/oshapio/intermediate-layer-generalization