mlsanityData to 7 Oct 2026

DICE: Leveraging Sparsification for Out-of-Distribution Detection

arXiv 2111.09805 ↗First posted 18 Nov 2021cs.LGcited by 166

Not on the map. Why?

Abstract (arXiv)

Detecting out-of-distribution (OOD) inputs is a central challenge for safely deploying machine learning models in the real world. Previous methods commonly rely on an OOD score derived from the overparameterized weight space, while largely overlooking the role of sparsification. In this paper, we reveal important insights that reliance on unimportant weights and units can directly attribute to the brittleness of OOD detection. To mitigate the issue, we propose a sparsification-based OOD detection framework termed DICE. Our key idea is to rank weights based on a measure of contribution, and selectively use the most salient weights to derive the output for OOD detection. We provide both empirical and theoretical insights, characterizing and explaining the mechanism by which DICE improves OOD detection. By pruning away noisy signals, DICE provably reduces the output variance for OOD data, resulting in a sharper output distribution and stronger separability from ID data. We demonstrate the effectiveness of sparsification-based OOD detection on several benchmarks and establish competitive performance.

Key facts. Cited by 166 papers in the map (data through 7 Oct 2026). Builds on 46 of its 71 references that are in the map. First posted on arXiv 18 Nov 2021.

Builds on

46 of 71 references.

References · 46

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  8. 2013Describing Textures in the Wildcited by 2,110
  9. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  10. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  11. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  12. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
34 more
  1. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  2. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  3. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  4. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  6. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  7. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  8. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  9. 2021Domain Generalization: A Surveycited by 760
  10. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 571
  11. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  12. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  13. 2015Towards Open Set Deep Networkscited by 533
  14. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  15. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  16. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  17. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  18. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  19. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  20. 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
  21. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  22. 2015TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Trackingcited by 165
  23. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  24. 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
  25. 2021Semantically Coherent Out-of-Distribution Detectioncited by 107
  26. 2019Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustnesscited by 104
  27. 2021Can multi-label classification networks know what they don't know?cited by 100
  28. 2021MOOD: Multi-level Out-of-distribution Detectioncited by 94
  29. 2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
  30. 2019Towards neural networks that provably know when they don't knowcited by 69
  31. 2021Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditionscited by 68
  32. 2018Discriminative out-of-distribution detection for semantic segmentationcited by 47
  33. 2019Learning Sparse Networks Using Targeted Dropoutcited by 40
  34. 2016NoiseOut: A Simple Way to Prune Neural Networkscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 45; 2024: 42; 2025: 51; 2026: 20.

166 papers.

Sort:

By area:

  1. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  2. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  3. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  4. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 178
  5. 2023LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learningcited by 107
  6. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  7. 2023Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancementcited by 77
  8. 2023Nearest Neighbor Guidance for Out-of-Distribution Detectioncited by 64
  9. 2023Fast Decision Boundary based Out-of-Distribution Detectorcited by 48
  10. 2023NECO: NEural Collapse Based Out-of-distribution detectioncited by 41
  11. 2024How Does Unlabeled Data Provably Help Out-of-Distribution Detection?cited by 36
  12. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
38 more
  1. 2023LINe: Out-of-Distribution Detection by Leveraging Important Neuronscited by 34
  2. 2024Outlier detection by ensembling uncertainty with negative objectnesscited by 34
  3. 2022OpenCon: Open-world Contrastive Learningcited by 33
  4. 2023Detecting Out-of-Distribution Through the Lens of Neural Collapsecited by 33
  5. 2022Block Selection Method for Using Feature Norm in Out-of-distribution Detectioncited by 30
  6. 2023Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detectioncited by 29
  7. 2023Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalizationcited by 28
  8. 2023Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researcherscited by 25
  9. 2024Kernel PCA for Out-of-Distribution Detectioncited by 24
  10. 2024ConjNorm: Tractable Density Estimation for Out-of-Distribution Detectioncited by 22
  11. 2024Out-of-Distribution Detection with a Single Unconditional Diffusion Modelcited by 22
  12. 2023GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradientscited by 21
  13. 2024Your Data Is Not Perfect: Towards Cross-Domain Out-of-Distribution Detection in Class-Imbalanced Datacited by 21
  14. 2023From Global to Local: Multi-scale Out-of-distribution Detectioncited by 18
  15. 2023DOS: Diverse Outlier Sampling for Out-of-Distribution Detectioncited by 16
  16. 2023Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capabilitycited by 16
  17. 2025Reliable Few-shot Learning under Dual Noisescited by 16
  18. 2023VRA: Variational Rectified Activation for Out-of-distribution Detectioncited by 15
  19. 2023Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Modelscited by 15
  20. 2023On the use of Mahalanobis distance for out-of-distribution detection with neural networks for medical imagingcited by 15
  21. 2023Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothnesscited by 15
  22. 2024Towards Optimal Feature-Shaping Methods for Out-of-Distribution Detectioncited by 13
  23. 2024A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?cited by 13
  24. 2024Gradient-Regularized Out-of-Distribution Detectioncited by 13
  25. 2024CLIPScope: Enhancing Zero-Shot OOD Detection with Bayesian Scoringcited by 13
  26. 2024Conjugated Semantic Pool Improves OOD Detection with Pre-trained Vision-Language Modelscited by 13
  27. 2023AUTO: Adaptive Outlier Optimization for Test-Time OOD Detectioncited by 12
  28. 2024Distribution Shifts at Scale: Out-of-distribution Detection in Earth Observationcited by 10
  29. 2023MCROOD: Multi-Class Radar Out-Of-Distribution Detectioncited by 9
  30. 2023Hybrid Energy Based Model in the Feature Space for Out-of-Distribution Detectioncited by 9
  31. 2023Activate and Reject: Towards Safe Domain Generalization under Category Shiftcited by 9
  32. 2023Unmasking the Chameleons: A Benchmark for Out-of-Distribution Detection in Medical Tabular Datacited by 8
  33. 2025CADRef: Robust Out-of-Distribution Detection via Class-Aware Decoupled Relative Feature Leveragingcited by 8
  34. 2022Rethinking Out-of-Distribution Detection From a Human-Centric Perspectivecited by 7
  35. 2023Out-of-Distribution Detection for Monocular Depth Estimationcited by 7
  36. 2024A Unified Approach Towards Active Learning and Out-of-Distribution Detectioncited by 7
  37. 2025AdaSCALE: Adaptive Scaling for OOD Detectioncited by 7
  38. 2024Category-Extensible Out-of-Distribution Detection via Hierarchical Context Descriptionscited by 6

50 most cited of 166 All 166 (JSON)