Data to 7 Oct 2026

Extremely Simple Activation Shaping for Out-of-Distribution Detection

arXiv 2209.09858 ↗First posted 20 Sep 2022cs.LG cs.CVcited by 212

Not on the map. Why?

Abstract (arXiv)

The separation between training and deployment of machine learning models implies that not all scenarios encountered in deployment can be anticipated during training, and therefore relying solely on advancements in training has its limits. Out-of-distribution (OOD) detection is an important area that stress-tests a model's ability to handle unseen situations: Do models know when they don't know? Existing OOD detection methods either incur extra training steps, additional data or make nontrivial modifications to the trained network. In contrast, in this work, we propose an extremely simple, post-hoc, on-the-fly activation shaping method, ASH, where a large portion (e.g. 90%) of a sample's activation at a late layer is removed, and the rest (e.g. 10%) simplified or lightly adjusted. The shaping is applied at inference time, and does not require any statistics calculated from training data. Experiments show that such a simple treatment enhances in-distribution and out-of-distribution distinction so as to allow state-of-the-art OOD detection on ImageNet, and does not noticeably deteriorate the in-distribution accuracy. Video, animation and code can be found at: https://andrijazz.github.io/ash

Key facts. Cited by 212 papers in the map (data through 7 Oct 2026). Builds on 43 of its 60 references that are in the map. First posted on arXiv 20 Sep 2022.

Builds on

43 of 60 references.

References · 43

2015Deep Residual Learning for Image Recognitioncited by 51,809
2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2020Language Models are Few-Shot Learnerscited by 32,199
2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
2016Densely Connected Convolutional Networkscited by 6,691
2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
2020Scaling Laws for Neural Language Modelscited by 6,411
2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
2013Intriguing properties of neural networkscited by 5,114
2017On Calibration of Modern Neural Networkscited by 4,860
2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416

31 more

2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
2022Locating and Editing Factual Associations in GPTcited by 2,609
2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
2013Describing Textures in the Wildcited by 2,110
2016Concrete Problems in AI Safetycited by 1,582
2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
2020Energy-based Out-of-distribution Detectioncited by 1,296
2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
2016Universal adversarial perturbationscited by 774
2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
2017Algorithmic decision making and the cost of fairnesscited by 402
2021Revisiting the Calibration of Modern Neural Networkscited by 387
2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
2019Explanations can be manipulated and geometry is to blamecited by 215
2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
2015TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Trackingcited by 165
2018Classification with Fairness Constraints: A Meta-Algorithm with Provable Guaranteescited by 146
2021Editing a classifier by rewriting its prediction rulescited by 64
2019How Can We Be So Dense? The Benefits of Using Highly Sparse Representationscited by 45
2019Learning Sparse Networks Using Targeted Dropoutcited by 40
2021A Near-Optimal Algorithm for Debiasing Trained Machine Learning Modelscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 38; 2024: 66; 2025: 60; 2026: 47.

212 papers.

Sort:

By area:

2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 283
2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 178
2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
2023Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancementcited by 77
2024Negative Label Guided OOD Detection with Pretrained Vision-Language Modelscited by 66
2025Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policiescited by 49
2023Fast Decision Boundary based Out-of-Distribution Detectorcited by 48
2023NECO: NEural Collapse Based Out-of-distribution detectioncited by 41
2023Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolationcited by 38
2023Learning to Augment Distributions for Out-of-Distribution Detectioncited by 38
2024How Does Unlabeled Data Provably Help Out-of-Distribution Detection?cited by 36

38 more

2025Failure Prediction at Runtime for Generative Robot Policiescited by 36
2024Outlier detection by ensembling uncertainty with negative objectnesscited by 34
2023Detecting Out-of-Distribution Through the Lens of Neural Collapsecited by 33
2023Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sourcescited by 31
2025Mahalanobis++: Improving OOD Detection via Feature Normalizationcited by 30
2023Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalizationcited by 28
2024MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalitiescited by 28
2024DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detectioncited by 28
2023DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalizationcited by 24
2024Kernel PCA for Out-of-Distribution Detectioncited by 24
2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 24
2023ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithmscited by 22
2024ConjNorm: Tractable Density Estimation for Out-of-Distribution Detectioncited by 22
2023GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradientscited by 21
2024Unexplored Faces of Robustness and Out-of-Distribution: Covariate Shifts in Environment and Sensor Domainscited by 21
2024LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Modelscited by 21
2025Secure On-Device Video OOD Detection Without Backpropagationcited by 19
2023A Baseline Analysis of Reward Models' Ability To Accurately Analyze Foundation Models Under Distribution Shiftcited by 17
2023Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capabilitycited by 16
2024AdaNeg: Adaptive Negative Proxy Guided OOD Detection with Vision-Language Modelscited by 16
2025OODD: Test-time Out-of-Distribution Detection with Dynamic Dictionarycited by 16
2022Detection of out-of-distribution samples using binary neuron activation patternscited by 15
2023VRA: Variational Rectified Activation for Out-of-distribution Detectioncited by 15
2023Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothnesscited by 15
2024Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detectioncited by 15
2025GOLD: Graph Out-of-Distribution Detection via Implicit Adversarial Latent Generationcited by 15
2023Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detectioncited by 14
2024FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detectioncited by 14
2024Towards Optimal Feature-Shaping Methods for Out-of-Distribution Detectioncited by 13
2024A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?cited by 13
2024Dissecting Out-of-Distribution Detection and Open-Set Recognition: A Critical Analysis of Methods and Benchmarkscited by 13
2025OpenMIBOOD: Open Medical Imaging Benchmarks for Out-Of-Distribution Detectioncited by 13
2023GAIA: Delving into Gradient-based Attribution Abnormality for Out-of-distribution Detectioncited by 12
2024Selective Classification Under Distribution Shiftscited by 12
2023Mixture of Gaussian-distributed Prototypes with Generative Modelling for Interpretable and Trustworthy Image Recognitioncited by 11
2025Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimizationcited by 11
2024Distribution Shifts at Scale: Out-of-distribution Detection in Earth Observationcited by 10
2023Unified Classification and Rejection: A One-versus-All Frameworkcited by 9

50 most cited of 212 All 212 (JSON)