# Boosting Operational DNN Testing Efficiency through Conditioning

arXiv: https://arxiv.org/abs/1906.02533 · HTML: https://mlsanity.com/paper/1906.02533/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2019.

## Built on by (most cited first, 22 of 22)

- [Black-Box Testing of Deep Neural Networks Through Test Case Diversity](https://mlsanity.com/paper/2112.12591/) (2021)
- [AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling](https://mlsanity.com/paper/2408.03573/) (2024)
- [Reliability Assessment and Safety Arguments for Machine Learning Components in System Assurance](https://mlsanity.com/paper/2112.00646/) (2021)
- [A Framework for Efficient Model Evaluation through Stratification, Sampling, and Estimation](https://mlsanity.com/paper/2406.07320/) (2024)
- [Assessing the Reliability of Deep Learning Classifiers Through Robustness Evaluation and Operational Profiles](https://mlsanity.com/paper/2106.01258/) (2021)
- [DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks](https://mlsanity.com/paper/2303.04878/) (2023)
- [Aries: Efficient Testing of Deep Neural Networks via Labeling-Free Accuracy Estimation](https://mlsanity.com/paper/2207.10942/) (2022)
- [Low-Shot Validation: Active Importance Sampling for Estimating Classifier Performance on Rare Categories](https://mlsanity.com/paper/2109.05720/) (2021)
- [Measuring Discrimination to Boost Comparative Testing for Multiple Deep Learning Models](https://mlsanity.com/paper/2103.04333/) (2021)
- [CodeS: Towards Code Model Generalization Under Distribution Shift](https://mlsanity.com/paper/2206.05480/) (2022)
- [Evaluating the Robustness of Test Selection Methods for Deep Neural Networks](https://mlsanity.com/paper/2308.01314/) (2023)
- [Iterative Assessment and Improvement of DNN Operational Accuracy](https://mlsanity.com/paper/2303.01295/) (2023)
- [Neuron Sensitivity Guided Test Case Selection for Deep Learning Testing](https://mlsanity.com/paper/2307.11011/) (2023)
- [Robust Active Learning: Sample-Efficient Training of Robust Deep Learning Models](https://mlsanity.com/paper/2112.02542/) (2021)
- [Characterizing and Understanding the Behavior of Quantized Models for Reliable Deployment](https://mlsanity.com/paper/2204.04220/) (2022)
- [DeepSample: DNN sampling-based testing for operational accuracy assessment](https://mlsanity.com/paper/2403.19271/) (2024)
- [LaF: Labeling-Free Model Selection for Automated Deep Neural Network Reusing](https://mlsanity.com/paper/2204.03994/) (2022)
- [Uncertainty Measurement of Deep Learning System based on the Convex Hull of Training Sets](https://mlsanity.com/paper/2405.16082/) (2024)
- [A Highly Efficient Diversity-based Input Selection for DNN Improvement Using VLMs](https://mlsanity.com/paper/2601.08024/) (2026)
- [DiffGAN: A Test Generation Approach for Differential Testing of Deep Neural Networks for Image Analysis](https://mlsanity.com/paper/2410.19794/) (2024)
- [Learning-Based Testing for Deep Learning: Enhancing Model Robustness with Adversarial Input Prioritization](https://mlsanity.com/paper/2509.23961/) (2025)
- [CAFD: Concept-Aware DNN Fault Detection using VLMs](https://mlsanity.com/paper/2605.24008/) (2026)

## Abstract (arXiv)

With the increasing adoption of Deep Neural Network (DNN) models as integral parts of software systems, efficient operational testing of DNNs is much in demand to ensure these models' actual performance in field conditions. A challenge is that the testing often needs to produce precise results with a very limited budget for labeling data collected in field. Viewing software testing as a practice of reliability estimation through statistical sampling, we re-interpret the idea behind conventional structural coverages as conditioning for variance reduction. With this insight we propose an efficient DNN testing method based on the conditioning on the representation learned by the DNN model under testing. The representation is defined by the probability distribution of the output of neurons in the last hidden layer of the model. To sample from this high dimensional distribution in which the operational data are sparsely distributed, we design an algorithm leveraging cross entropy minimization. Experiments with various DNN models and datasets were conducted to evaluate the general efficiency of the approach. The results show that, compared with simple random sampling, this approach requires only about a half of labeled inputs to achieve the same level of precision.