mlsanityData to 7 Oct 2026

Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability

arXiv 2306.03715 ↗First posted 6 Jun 2023cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications. Previous paradigms either explore better scoring functions or utilize the knowledge of outliers to equip the models with the ability of OOD detection. However, few of them pay attention to the intrinsic OOD detection capability of the given model. In this work, we generally discover the existence of an intermediate stage of a model trained on in-distribution (ID) data having higher OOD detection performance than that of its final stage across different settings, and further identify one critical data-level attribution to be learning with the atypical samples. Based on such insights, we propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data. Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them. Extensive experiments and analysis demonstrate the effectiveness of our method. The code is available at: https://github.com/tmlr-group/Unleashing-Mask.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 47 of its 71 references that are in the map. First posted on arXiv 6 Jun 2023.

Builds on

47 of 71 references.

References · 47

  1. 2016Densely Connected Convolutional Networkscited by 6,691
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Wide Residual Networkscited by 3,057
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2013Describing Textures in the Wildcited by 2,110
  8. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  9. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  10. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  11. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  12. 2020Energy-based Out-of-distribution Detectioncited by 1,296
35 more
  1. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  2. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  3. 2017A Closer Look at Memorization in Deep Networkscited by 936
  4. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  5. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  6. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  7. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  8. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  9. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  10. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  11. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  12. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
  13. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  14. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 301
  15. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  16. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  17. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  18. 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
  19. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  20. 2019What's Hidden in a Randomly Weighted Neural Network?cited by 236
  21. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  22. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  23. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  24. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  25. 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
  26. 2019Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancycited by 112
  27. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  28. 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
  29. 2021MOOD: Multi-level Out-of-distribution Detectioncited by 94
  30. 2022Training OOD Detectors in their Natural Habitatscited by 86
  31. 2022Pathways: Asynchronous Distributed Dataflow for MLcited by 81
  32. 2016Places: An Image Database for Deep Scene Understandingcited by 76
  33. 2020Do We Need Zero Training Loss After Achieving Zero Training Error?cited by 61
  34. 2023Out-of-distribution Detection with Implicit Outlier Transformationcited by 42
  35. 2020Embedded Encoder-Decoder in Convolutional Networks Towards Explainable AIcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 7; 2025: 2; 2026: 3.

16 papers.

Sort:

By area:

  1. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  2. 2023Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolationcited by 38
  3. 2023Learning to Augment Distributions for Out-of-Distribution Detectioncited by 38
  4. 2024How Does Unlabeled Data Provably Help Out-of-Distribution Detection?cited by 36
  5. 2024Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detectioncited by 30
  6. 2024Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detectioncited by 15
  7. 2024When and How Does In-Distribution Label Help Out-of-Distribution Detection?cited by 6
  8. 2024Typicalness-Aware Learning for Failure Detectioncited by 6
  9. 2024Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Studycited by 5
  10. 2024What If the Input is Expanded in OOD Detection?cited by 4
  11. 2025Outlier Synthesis via Hamiltonian Monte Carlo for Out-of-Distribution Detectioncited by 4
  12. 2023Distilling the Unknown to Unveil Certaintycited by 0
4 more
  1. 2025What Can We Learn from Harry Potter? An Exploratory Study of Visual Representation Learning from Atypical Videoscited by 0
  2. 2026Vision Also You Need: Navigating Out-of-Distribution Detection with Multimodal Large Language Modelcited by 0
  3. 2026Shaping Parameter Contribution Patterns for Out-of-Distribution Detectioncited by 0
  4. 2026AOE: Exhaustive Out-of-Distribution Detection via Recalibrating Outlier Labelscited by 0