# Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancement

arXiv: https://arxiv.org/abs/2310.00227 · HTML: https://mlsanity.com/paper/2310.00227/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 30 Sep 2023.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [Scaling Out-of-Distribution Detection for Real-World Settings](https://mlsanity.com/paper/1911.11132/) (2019)
- [ViM: Out-Of-Distribution with Virtual-logit Matching](https://mlsanity.com/paper/2203.10807/) (2022)
- [Open-Set Recognition: a Good Closed-Set Classifier is All You Need?](https://mlsanity.com/paper/2110.06207/) (2021)
- [OpenOOD: Benchmarking Generalized Out-of-Distribution Detection](https://mlsanity.com/paper/2210.07242/) (2022)
- [Mitigating Neural Network Overconfidence with Logit Normalization](https://mlsanity.com/paper/2205.09310/) (2022)
- [Learning Confidence for Out-of-Distribution Detection in Neural Networks](https://mlsanity.com/paper/1802.04865/) (2018)
- [Extremely Simple Activation Shaping for Out-of-Distribution Detection](https://mlsanity.com/paper/2209.09858/) (2022)
- [A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection](https://mlsanity.com/paper/2106.09022/) (2021)
- [OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection](https://mlsanity.com/paper/2306.09301/) (2023)
- [DICE: Leveraging Sparsification for Out-of-Distribution Detection](https://mlsanity.com/paper/2111.09805/) (2021)
- [TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking](https://mlsanity.com/paper/1504.06755/) (2015)
- [In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation](https://mlsanity.com/paper/2306.00826/) (2023)
- [How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?](https://mlsanity.com/paper/2203.04450/) (2022)
- [The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers](https://mlsanity.com/paper/2210.06313/) (2022)
- [SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer](https://mlsanity.com/paper/2303.17605/) (2023)
- [GACT: Activation Compressed Training for Generic Network Architectures](https://mlsanity.com/paper/2206.11357/) (2022)
- [DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training](https://mlsanity.com/paper/2202.13808/) (2022)

## Built on by (most cited first, 30 of 77)

- [Fast Decision Boundary based Out-of-Distribution Detector](https://mlsanity.com/paper/2312.11536/) (2023)
- [Detecting Out-of-Distribution Through the Lens of Neural Collapse](https://mlsanity.com/paper/2311.01479/) (2023)
- [Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advances](https://mlsanity.com/paper/2409.11884/) (2024)
- [Kernel PCA for Out-of-Distribution Detection](https://mlsanity.com/paper/2402.02949/) (2024)
- [LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2407.08966/) (2024)
- [Reliable Few-shot Learning under Dual Noises](https://mlsanity.com/paper/2506.16330/) (2025)
- [AdaNeg: Adaptive Negative Proxy Guided OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2410.20149/) (2024)
- [Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detection](https://mlsanity.com/paper/2411.03359/) (2024)
- [Energy-based Hopfield Boosting for Out-of-Distribution Detection](https://mlsanity.com/paper/2405.08766/) (2024)
- [CADRef: Robust Out-of-Distribution Detection via Class-Aware Decoupled Relative Feature Leveraging](https://mlsanity.com/paper/2503.00325/) (2025)
- [A Unified Approach Towards Active Learning and Out-of-Distribution Detection](https://mlsanity.com/paper/2405.11337/) (2024)
- [AdaSCALE: Adaptive Scaling for OOD Detection](https://mlsanity.com/paper/2503.08023/) (2025)
- [Your Classifier Can Be Secretly a Likelihood-Based OOD Detector](https://mlsanity.com/paper/2408.04851/) (2024)
- [Outlier Synthesis via Hamiltonian Monte Carlo for Out-of-Distribution Detection](https://mlsanity.com/paper/2501.16718/) (2025)
- [Improving Out-of-Distribution Detection via Dynamic Covariance Calibration](https://mlsanity.com/paper/2506.09399/) (2025)
- [A Geometry-Based View of Mahalanobis OOD Detection](https://mlsanity.com/paper/2510.15202/) (2025)
- [Logit Scaling for Out-of-Distribution Detection](https://mlsanity.com/paper/2409.01175/) (2024)
- [TS-OOD: Evaluating Time-Series Out-of-Distribution Detection and Prospective Directions for Progress](https://mlsanity.com/paper/2502.15901/) (2025)
- [Enhancing Out-of-Distribution Detection with Extended Logit Normalization](https://mlsanity.com/paper/2504.11434/) (2025)
- [Out-of-Distribution Detection using Neural Activation Prior](https://mlsanity.com/paper/2402.18162/) (2024)
- [Deciphering the Definition of Adversarial Robustness for post-hoc OOD Detectors](https://mlsanity.com/paper/2406.15104/) (2024)
- [Improving Out-of-Distribution Detection by Combining Existing Post-hoc Methods](https://mlsanity.com/paper/2407.07135/) (2024)
- [Out-of-Distribution Detection with Relative Angles](https://mlsanity.com/paper/2410.04525/) (2024)
- [Reflexive Guidance: Improving OoDD in Vision-Language Models via Self-Guided Image-Adaptive Concept Generation](https://mlsanity.com/paper/2410.14975/) (2024)
- [OpenEarthSensing: Large-Scale Fine-Grained Benchmark for Open-World Remote Sensing](https://mlsanity.com/paper/2502.20668/) (2025)
- [Revisiting Out-of-Distribution Detection in Real-time Object Detection: From Benchmark Pitfalls to a New Mitigation Paradigm](https://mlsanity.com/paper/2503.07330/) (2025)
- [STOOD-X methodology: using statistical nonparametric test for OOD Detection Large-Scale datasets enhanced with explainability](https://mlsanity.com/paper/2504.02685/) (2025)
- [Buffer-free Class-Incremental Learning with Out-of-Distribution Detection](https://mlsanity.com/paper/2505.23412/) (2025)
- [Spurious-Aware Prototype Refinement for Reliable Out-of-Distribution Detection](https://mlsanity.com/paper/2506.23881/) (2025)
- [Activation Subspaces for Out-of-Distribution Detection](https://mlsanity.com/paper/2508.21695/) (2025)

## Abstract (arXiv)

The capacity of a modern deep learning system to determine if a sample falls within its realm of knowledge is fundamental and important. In this paper, we offer insights and analyses of recent state-of-the-art out-of-distribution (OOD) detection methods - extremely simple activation shaping (ASH). We demonstrate that activation pruning has a detrimental effect on OOD detection, while activation scaling enhances it. Moreover, we propose SCALE, a simple yet effective post-hoc network enhancement method for OOD detection, which attains state-of-the-art OOD detection performance without compromising in-distribution (ID) accuracy. By integrating scaling concepts into the training process to capture a sample's ID characteristics, we propose Intermediate Tensor SHaping (ISH), a lightweight method for training time OOD detection enhancement. We achieve AUROC scores of +1.85% for near-OOD and +0.74% for far-OOD datasets on the OpenOOD v1.5 ImageNet-1K benchmark. Our code and models are available at https://github.com/kai422/SCALE.