mlsanityData to 7 Oct 2026

Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection

arXiv 2303.01201 ↗First posted 2 Mar 2023cs.LG cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Detecting Out-of-distribution (OOD) inputs have been a critical issue for neural networks in the open world. However, the unstable behavior of OOD detection along the optimization trajectory during training has not been explored clearly. In this paper, we first find the performance of OOD detection suffers from overfitting and instability during training: 1) the performance could decrease when the training error is near zero, and 2) the performance would vary sharply in the final stage of training. Based on our findings, we propose Average of Pruning (AoP), consisting of model averaging and pruning, to mitigate the unstable behaviors. Specifically, model averaging can help achieve a stable performance by smoothing the landscape, and pruning is certified to eliminate the overfitting by eliminating redundant features. Comprehensive experiments on various datasets and architectures are conducted to verify the effectiveness of our method.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 58 of its 82 references that are in the map. First posted on arXiv 2 Mar 2023.

Builds on

58 of 82 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  7. 2016Wide Residual Networkscited by 3,057
  8. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  10. 2013Describing Textures in the Wildcited by 2,110
  11. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  12. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
38 more
  1. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  2. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  3. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  4. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  5. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  6. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  7. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  8. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  9. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  10. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 627
  11. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  12. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  13. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  14. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
  15. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  16. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  17. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  18. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
  19. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  20. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  21. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  22. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  23. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  24. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  25. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  26. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  27. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  28. 2015TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Trackingcited by 165
  29. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  30. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  31. 2020Confidence-Aware Learning for Deep Neural Networkscited by 113
  32. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  33. 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
  34. 2018Bias-Reduced Uncertainty Estimation for Deep Neural Classifierscited by 106
  35. 2021AugMax: Adversarial Composition of Random Augmentations for Robust Trainingcited by 89
  36. 2021Sparse Training via Boosting Pruning Plasticity with Neuroregenerationcited by 83
  37. 2020Gradient Centralization: A New Optimization Technique for Deep Neural Networkscited by 82
  38. 2021Provable Guarantees for Understanding Out-of-distribution Detectioncited by 78

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 5; 2025: 2; 2026: 3.

14 papers.

Sort:

By area:

  1. 2023Deep Model Fusion: A Surveycited by 89
  2. 2023OpenMix: Exploring Outlier Samples for Misclassification Detectioncited by 38
  3. 2024Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detectioncited by 14
  4. 2023Unified Classification and Rejection: A One-versus-All Frameworkcited by 9
  5. 2024Revisiting Confidence Estimation: Towards Reliable Failure Predictioncited by 9
  6. 2024PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learningcited by 7
  7. 2024Optimal Parameter and Neuron Pruning for Out-of-Distribution Detectioncited by 4
  8. 2023A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifierscited by 1
  9. 2024Embracing Unknown Step by Step: Towards Reliable Sparse Training in Real Worldcited by 1
  10. 2025Towards Efficient and General-Purpose Few-Shot Misclassification Detection for Vision-Language Modelscited by 1
  11. 2025TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Datacited by 0
  12. 2026The Invisible Gorilla Effect in Out-of-distribution Detectioncited by 0
2 more
  1. 2026Optimal Stability of KL Divergence under Gaussian Perturbationscited by 0
  2. 2026HAND: A Biologically-Inspired Activation Function that Improves Generalisation and Sample Efficiency in Image Classificationcited by 0