# A Review and Refinement of Surprise Adequacy

arXiv: https://arxiv.org/abs/2103.05939 · HTML: https://mlsanity.com/paper/2103.05939/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 10 Mar 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [MNIST-C: A Robustness Benchmark for Computer Vision](https://mlsanity.com/paper/1906.02337/) (2019)
- [Guiding Deep Learning System Testing using Surprise Adequacy](https://mlsanity.com/paper/1808.08444/) (2018)
- [Misbehaviour Prediction for Autonomous Driving Systems](https://mlsanity.com/paper/1910.04443/) (2019)
- [Test Selection for Deep Learning Systems](https://mlsanity.com/paper/1904.13195/) (2019)
- [Importance-Driven Deep Learning System Testing](https://mlsanity.com/paper/2002.03433/) (2020)
- [Reducing DNN Labelling Cost using Surprise Adequacy: An Industrial Case Study for Autonomous Driving](https://mlsanity.com/paper/2006.00894/) (2020)
- [Input Prioritization for Testing Neural Networks](https://mlsanity.com/paper/1901.03768/) (2019)
- [Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring](https://mlsanity.com/paper/2102.00902/) (2021)
- [There is Limited Correlation between Coverage and Robustness for Deep Neural Networks](https://mlsanity.com/paper/1911.05904/) (2019)
- [RAID: Randomized Adversarial-Input Detection for Neural Networks](https://mlsanity.com/paper/2002.02776/) (2020)
- [Operational Calibration: Debugging Confidence Errors for DNNs in the Field](https://mlsanity.com/paper/1910.02352/) (2019)

## Built on by (most cited first, 9 of 9)

- [Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)](https://mlsanity.com/paper/2205.00664/) (2022)
- [Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testing](https://mlsanity.com/paper/2207.10495/) (2022)
- [Uncertainty Quantification for Deep Neural Networks: An Empirical Comparison and Usage Guidelines](https://mlsanity.com/paper/2212.07118/) (2022)
- [FAST: Boosting Uncertainty-based Test Prioritization Methods for Neural Networks via Feature Selection](https://mlsanity.com/paper/2409.09130/) (2024)
- [MetaSel: A Test Selection Approach for Fine-tuned DNN Models](https://mlsanity.com/paper/2503.17534/) (2025)
- [Guiding the retraining of convolutional neural networks against adversarial inputs](https://mlsanity.com/paper/2207.03689/) (2022)
- [CheapET-3: Cost-Efficient Use of Remote DNN Models](https://mlsanity.com/paper/2208.11552/) (2022)
- [Self-Emotion-Mediated Exploration in Artificial Intelligence Mirrors: Findings from Cognitive Psychology](https://mlsanity.com/paper/2302.06615/) (2023)
- [Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networks](https://mlsanity.com/paper/2304.02654/) (2023)

## Abstract (arXiv)

Surprise Adequacy (SA) is one of the emerging and most promising adequacy criteria for Deep Learning (DL) testing. As an adequacy criterion, it has been used to assess the strength of DL test suites. In addition, it has also been used to find inputs to a Deep Neural Network (DNN) which were not sufficiently represented in the training data, or to select samples for DNN retraining. However, computation of the SA metric for a test suite can be prohibitively expensive, as it involves a quadratic number of distance calculations. Hence, we developed and released a performance-optimized, but functionally equivalent, implementation of SA, reducing the evaluation time by up to 97%. We also propose refined variants of the SA omputation algorithm, aiming to further increase the evaluation speed. We then performed an empirical study on MNIST, focused on the out-of-distribution detection capabilities of SA, which allowed us to reproduce parts of the results presented when SA was first released. The experiments show that our refined variants are substantially faster than plain SA, while producing comparable outcomes. Our experimental results exposed also an overlooked issue of SA: it can be highly sensitive to the non-determinism associated with the DNN training procedure.