# Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty

arXiv: https://arxiv.org/abs/2004.11573 · HTML: https://mlsanity.com/paper/2004.11573/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2020.

## Built on by (most cited first, 26 of 26)

- [Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models](https://mlsanity.com/paper/2307.10236/) (2023)
- [Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)](https://mlsanity.com/paper/2205.00664/) (2022)
- [NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks](https://mlsanity.com/paper/2203.12915/) (2022)
- [When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study](https://mlsanity.com/paper/2212.11368/) (2022)
- [DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Score](https://mlsanity.com/paper/2109.07514/) (2021)
- [PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing](https://mlsanity.com/paper/2111.10481/) (2021)
- [Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring](https://mlsanity.com/paper/2102.00902/) (2021)
- [Aries: Efficient Testing of Deep Neural Networks via Labeling-Free Accuracy Estimation](https://mlsanity.com/paper/2207.10942/) (2022)
- [Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testing](https://mlsanity.com/paper/2207.10495/) (2022)
- [Beyond Voxel Prediction Uncertainty: Identifying brain lesions you can trust](https://mlsanity.com/paper/2209.10877/) (2022)
- [Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward](https://mlsanity.com/paper/2404.08517/) (2024)
- [An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applications](https://mlsanity.com/paper/2301.13476/) (2023)
- [Repairing Deep Neural Networks Based on Behavior Imitation](https://mlsanity.com/paper/2305.03365/) (2023)
- [Evaluating the Robustness of Test Selection Methods for Deep Neural Networks](https://mlsanity.com/paper/2308.01314/) (2023)
- [An Exploratory Study of AI System Risk Assessment from the Lens of Data Distribution and Uncertainty](https://mlsanity.com/paper/2212.06828/) (2022)
- [Uncertainty Quantification for Deep Neural Networks: An Empirical Comparison and Usage Guidelines](https://mlsanity.com/paper/2212.07118/) (2022)
- [GGT: Graph-Guided Testing for Adversarial Sample Detection of Deep Neural Network](https://mlsanity.com/paper/2107.07043/) (2021)
- [An Empirical Study on Bugs Inside PyTorch: A Replication Study](https://mlsanity.com/paper/2307.13777/) (2023)
- [Generalizing Neural Networks by Reflecting Deviating Data in Production](https://mlsanity.com/paper/2110.02718/) (2021)
- [Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networks](https://mlsanity.com/paper/2304.02654/) (2023)
- [Relationship between Uncertainty in DNNs and Adversarial Attacks](https://mlsanity.com/paper/2409.13232/) (2024)
- [TopoMap: A Feature-based Semantic Discriminator of the Topographical Regions in the Test Input Space](https://mlsanity.com/paper/2509.03242/) (2025)
- [Learning-Based Testing for Deep Learning: Enhancing Model Robustness with Adversarial Input Prioritization](https://mlsanity.com/paper/2509.23961/) (2025)
- [As If We've Met Before: LLMs Exhibit Certainty in Recognizing Seen Files](https://mlsanity.com/paper/2511.15192/) (2025)
- [Code-MUE: Measuring Code LLMs' Uncertainty through Execution-based Semantic Interaction Graphs](https://mlsanity.com/paper/2607.12273/) (2026)
- [Test Case Prioritization for DNNs via Neural Collapse Instability](https://mlsanity.com/paper/2607.20046/) (2026)

## Abstract (arXiv)

Over the past decade, deep learning (DL) has been successfully applied to many industrial domain-specific tasks. However, the current state-of-the-art DL software still suffers from quality issues, which raises great concern especially in the context of safety- and security-critical scenarios. Adversarial examples (AEs) represent a typical and important type of defects needed to be urgently addressed, on which a DL software makes incorrect decisions. Such defects occur through either intentional attack or physical-world noise perceived by input sensors, potentially hindering further industry deployment. The intrinsic uncertainty nature of deep learning decisions can be a fundamental reason for its incorrect behavior. Although some testing, adversarial attack and defense techniques have been recently proposed, it still lacks a systematic study to uncover the relationship between AEs and DL uncertainty. In this paper, we conduct a large-scale study towards bridging this gap. We first investigate the capability of multiple uncertainty metrics in differentiating benign examples (BEs) and AEs, which enables to characterize the uncertainty patterns of input data. Then, we identify and categorize the uncertainty patterns of BEs and AEs, and find that while BEs and AEs generated by existing methods do follow common uncertainty patterns, some other uncertainty patterns are largely missed. Based on this, we propose an automated testing technique to generate multiple types of uncommon AEs and BEs that are largely missed by existing techniques. Our further evaluation reveals that the uncommon data generated by our method is hard to be defended by the existing defense techniques with the average defense success rate reduced by 35%. Our results call for attention and necessity to generate more diverse data for evaluating quality assurance solutions of DL software.