# Aries: Efficient Testing of Deep Neural Networks via Labeling-Free Accuracy Estimation

arXiv: https://arxiv.org/abs/2207.10942 · HTML: https://mlsanity.com/paper/2207.10942/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 22 Jul 2022.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars](https://mlsanity.com/paper/1708.08559/) (2017)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [Adversarial Active Learning for Deep Networks: a Margin Based Approach](https://mlsanity.com/paper/1802.09841/) (2018)
- [PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures](https://mlsanity.com/paper/2112.05135/) (2021)
- [Guiding Deep Learning System Testing using Surprise Adequacy](https://mlsanity.com/paper/1808.08444/) (2018)
- [Globally-Robust Neural Networks](https://mlsanity.com/paper/2102.08452/) (2021)
- [Are Labels Always Necessary for Classifier Accuracy Evaluation?](https://mlsanity.com/paper/2007.02915/) (2020)
- [Learnable Boundary Guided Adversarial Training](https://mlsanity.com/paper/2011.11164/) (2020)
- [Towards Playing Full MOBA Games with Deep Reinforcement Learning](https://mlsanity.com/paper/2011.12692/) (2020)
- [DLFuzz: Differential Fuzzing Testing of Deep Learning Systems](https://mlsanity.com/paper/1808.09413/) (2018)
- [DeepID3: Face Recognition with Very Deep Neural Networks](https://mlsanity.com/paper/1502.00873/) (2015)
- [Reducing Label Effort: Self-Supervised meets Active Learning](https://mlsanity.com/paper/2108.11458/) (2021)
- [Test Selection for Deep Learning Systems](https://mlsanity.com/paper/1904.13195/) (2019)
- [Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty](https://mlsanity.com/paper/2004.11573/) (2020)
- [Boosting Operational DNN Testing Efficiency through Conditioning](https://mlsanity.com/paper/1906.02533/) (2019)
- [DeepEvolution: A Search-Based Testing Approach for Deep Neural Networks](https://mlsanity.com/paper/1909.02563/) (2019)
- [What Are We Really Testing in Mutation Testing for Machine Learning? A Critical Reflection](https://mlsanity.com/paper/2103.01341/) (2021)

## Built on by (most cited first, 9 of 9)

- [VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2409.12894/) (2024)
- [AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling](https://mlsanity.com/paper/2408.03573/) (2024)
- [Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities](https://mlsanity.com/paper/2212.10017/) (2022)
- [Curved Representation Space of Vision Transformers](https://mlsanity.com/paper/2210.05742/) (2022)
- [Suitability Filter: A Statistical Framework for Classifier Evaluation in Real-World Deployment Settings](https://mlsanity.com/paper/2505.22356/) (2025)
- [A3Rank: Augmentation Alignment Analysis for Prioritizing Overconfident Failing Samples for Deep Learning Models](https://mlsanity.com/paper/2407.14114/) (2024)
- [On Accelerating Deep Neural Network Mutation Analysis by Neuron and Mutant Clustering](https://mlsanity.com/paper/2501.12598/) (2025)
- [LeCov: Multi-level Testing Criteria for Large Language Models](https://mlsanity.com/paper/2408.10474/) (2024)
- [Test Case Prioritization for DNNs via Neural Collapse Instability](https://mlsanity.com/paper/2607.20046/) (2026)

## Abstract (arXiv)

Deep learning (DL) plays a more and more important role in our daily life due to its competitive performance in industrial application domains. As the core of DL-enabled systems, deep neural networks (DNNs) need to be carefully evaluated to ensure the produced models match the expected requirements. In practice, the de facto standard to assess the quality of DNNs in the industry is to check their performance (accuracy) on a collected set of labeled test data. However, preparing such labeled data is often not easy partly because of the huge labeling effort, i.e., data labeling is labor-intensive, especially with the massive new incoming unlabeled data every day. Recent studies show that test selection for DNN is a promising direction that tackles this issue by selecting minimal representative data to label and using these data to assess the model. However, it still requires human effort and cannot be automatic. In this paper, we propose a novel technique, named textitAries, that can estimate the performance of DNNs on new unlabeled data using only the information obtained from the original test data. The key insight behind our technique is that the model should have similar prediction accuracy on the data which have similar distances to the decision boundary. We performed a large-scale evaluation of our technique on two famous datasets, CIFAR-10 and Tiny-ImageNet, four widely studied DNN models including ResNet101 and DenseNet121, and 13 types of data transformation methods. Results show that the estimated accuracy by textitAries is only 0.03% -- 2.60% off the true accuracy. Besides, textitAries also outperforms the state-of-the-art labeling-free methods in 50 out of 52 cases and selection-labeling-based methods in 96 out of 128 cases.