mlsanityData to 7 Oct 2026

Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)

arXiv 2205.00664 ↗First posted 2 May 2022cs.LG cs.AI cs.SEcited by 27

Not on the map. Why?

Abstract (arXiv)

Test Input Prioritizers (TIP) for Deep Neural Networks (DNN) are an important technique to handle the typically very large test datasets efficiently, saving computation and labeling costs. This is particularly true for large-scale, deployed systems, where inputs observed in production are recorded to serve as potential test or training data for the next versions of the system. Feng et. al. propose DeepGini, a very fast and simple TIP, and show that it outperforms more elaborate techniques such as neuron- and surprise coverage. In a large-scale study (4 case studies, 8 test datasets, 32'200 trained models) we verify their findings. However, we also find that other comparable or even simpler baselines from the field of uncertainty quantification, such as the predicted softmax likelihood or the entropy of the predicted softmax likelihoods perform equally well as DeepGini.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 19 of its 47 references that are in the map. First posted on arXiv 2 May 2022.

Builds on

19 of 47 references.

References · 19

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  6. 2017Deep Bayesian Active Learning with Image Datacited by 812
  7. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  8. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  9. 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 307
  10. 2018DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systemscited by 144
  11. 2019MNIST-C: A Robustness Benchmark for Computer Visioncited by 137
  12. 2018Guiding Deep Learning System Testing using Surprise Adequacycited by 123
7 more
  1. 2019Misbehaviour Prediction for Autonomous Driving Systemscited by 55
  2. 2019Test Selection for Deep Learning Systemscited by 27
  3. 2020Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertaintycited by 26
  4. 2020Reducing DNN Labelling Cost using Surprise Adequacy: An Industrial Case Study for Autonomous Drivingcited by 21
  5. 2021Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoringcited by 11
  6. 2020Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantificationcited by 9
  7. 2021A Review and Refinement of Surprise Adequacycited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 10; 2024: 4; 2025: 4; 2026: 4.

27 papers.

Sort:

  1. 2023FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clusteringcited by 15
  2. 2024FAN: Fourier Analysis Networkscited by 12
  3. 2023DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networkscited by 11
  4. 2022Hierarchical Distribution-Aware Testing of Deep Learningcited by 8
  5. 2022Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testingcited by 8
  6. 2023Evaluating the Robustness of Test Selection Methods for Deep Neural Networkscited by 4
  7. 2025Benchmarking Image Perturbations for Testing Automated Driving Assistance Systemscited by 4
  8. 2022To Softmax, or not to Softmax: that is the question when applying Active Learning for Transformer Modelscited by 3
  9. 2023Neuron Sensitivity Guided Test Case Selection for Deep Learning Testingcited by 3
  10. 2024FAST: Boosting Uncertainty-based Test Prioritization Methods for Neural Networks via Feature Selectioncited by 3
  11. 2025MetaSel: A Test Selection Approach for Fine-tuned DNN Modelscited by 3
  12. 2023Contrastive inverse regression for dimension reductioncited by 2
15 more
  1. 2022Guiding the retraining of convolutional neural networks against adversarial inputscited by 1
  2. 2022CheapET-3: Cost-Efficient Use of Remote DNN Modelscited by 1
  3. 2023VisAlign: Dataset for Measuring the Degree of Alignment between AI and Humans in Visual Perceptioncited by 1
  4. 2023A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifierscited by 1
  5. 2024Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deploymentcited by 1
  6. 2024Evaluation and Improvement of Fault Detection for Large Language Modelscited by 1
  7. 2025Toward Efficient Testing of Graph Neural Networks via Test Input Prioritizationcited by 1
  8. 2026A Highly Efficient Diversity-based Input Selection for DNN Improvement Using VLMscited by 1
  9. 2026TestifAI: Tomography-Based Testing for Deep Learning Systemscited by 1
  10. 2023Neuroevolutionary algorithms driven by neuron coverage metrics for semi-supervised classificationcited by 0
  11. 2023Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networkscited by 0
  12. 2023Hazards in Deep Learning Testing: Prevalence, Impact and Recommendationscited by 0
  13. 2025Learning-Based Testing for Deep Learning: Enhancing Model Robustness with Adversarial Input Prioritizationcited by 0
  14. 2026CAFD: Concept-Aware DNN Fault Detection using VLMscited by 0
  15. 2026Test Case Prioritization for DNNs via Neural Collapse Instabilitycited by 0