mlsanityData to 7 Oct 2026

CertPri: Certifiable Prioritization for Deep Neural Networks via Movement Cost in Feature Space

arXiv 2307.09375 ↗First posted 18 Jul 2023cs.SE cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) have demonstrated their outperformance in various software systems, but also exhibit misbehavior and even result in irreversible disasters. Therefore, it is crucial to identify the misbehavior of DNN-based software and improve DNNs' quality. Test input prioritization is one of the most appealing ways to guarantee DNNs' quality, which prioritizes test inputs so that more bug-revealing inputs can be identified earlier with limited time and manual labeling efforts. However, the existing prioritization methods are still limited from three aspects: certifiability, effectiveness, and generalizability. To overcome the challenges, we propose CertPri, a test input prioritization technique designed based on a movement cost perspective of test inputs in DNNs' feature space. CertPri differs from previous works in three key aspects: (1) certifiable: it provides a formal robustness guarantee for the movement cost; (2) effective: it leverages formally guaranteed movement costs to identify malicious bug-revealing inputs; and (3) generic: it can be applied to various tasks, data, models, and scenarios. Extensive evaluations across 2 tasks (i.e., classification and regression), 6 data forms, 4 model structures, and 2 scenarios (i.e., white-box and black-box) demonstrate CertPri's superior performance. For instance, it significantly improves 53.97% prioritization effectiveness on average compared with baselines. Its robustness and generalizability are 1.41~2.00 times and 1.33~3.39 times that of baselines on average, respectively.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 35 of its 74 references that are in the map. First posted on arXiv 18 Jul 2023.

Builds on

35 of 74 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  8. 2016Adversarial examples in the physical worldcited by 1,762
  9. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  10. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  11. 2018Deep Learning on Graphs: A Surveycited by 393
  12. 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 307
23 more
  1. 2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 249
  2. 2018Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approachcited by 181
  3. 2018DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systemscited by 144
  4. 2018Guiding Deep Learning System Testing using Surprise Adequacycited by 123
  5. 2018Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attackcited by 123
  6. 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
  7. 2018Semantic Adversarial Examplescited by 86
  8. 2020An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Modelscited by 67
  9. 2018DLFuzz: Differential Fuzzing Testing of Deep Learning Systemscited by 64
  10. 2019Adversarial Defense by Restricting the Hidden Space of Deep Neural Networkscited by 56
  11. 2017Feature-Guided Black-Box Safety Testing of Deep Neural Networkscited by 48
  12. 2021DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injectioncited by 42
  13. 2021NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identificationcited by 42
  14. 2019POPQORN: Quantifying Robustness of Recurrent Neural Networkscited by 36
  15. 2018Adversarial Defense by Stratified Convolutional Sparse Codingcited by 31
  16. 2019DeepFault: Fault Localization for Deep Neural Networkscited by 29
  17. 2019Test Selection for Deep Learning Systemscited by 27
  18. 2018A comparison of deep networks with ReLU activation function and linear spline-type methodscited by 18
  19. 2022NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networkscited by 17
  20. 2019Input Prioritization for Testing Neural Networkscited by 11
  21. 2021DeepPoison: Feature Transfer Based Stealthy Poisoning Attackcited by 8
  22. 2017Empirical Evaluation of Mutation-based Test Prioritization Techniquescited by 4
  23. 2019A Noise-Sensitivity-Analysis-Based Test Prioritization Technique for Deep Neural Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 2.

4 papers.

  1. 2024FAST: Boosting Uncertainty-based Test Prioritization Methods for Neural Networks via Feature Selectioncited by 3
  2. 2024Evaluation and Improvement of Fault Detection for Large Language Modelscited by 1
  3. 2026Learnable Weighting of Intra-Attribute Distances for Categorical Data Clustering with Nominal and Ordinal Attributescited by 0
  4. 2026Test Case Prioritization for DNNs via Neural Collapse Instabilitycited by 0