# Assessing the Reliability of Deep Learning Classifiers Through Robustness Evaluation and Operational Profiles

arXiv: https://arxiv.org/abs/2106.01258 · HTML: https://mlsanity.com/paper/2106.01258/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 57 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Safety Verification of Deep Neural Networks](https://mlsanity.com/paper/1610.06940/) (2016)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [A Tutorial on Kernel Density Estimation and Recent Advances](https://mlsanity.com/paper/1704.03924/) (2017)
- [A Statistical Approach to Assessing Neural Network Robustness](https://mlsanity.com/paper/1811.07209/) (2018)
- [Boosting Operational DNN Testing Efficiency through Conditioning](https://mlsanity.com/paper/1906.02533/) (2019)
- [A Safety Framework for Critical Systems Utilising Deep Neural Networks](https://mlsanity.com/paper/2003.05311/) (2020)
- [Coverage Guided Testing for Recurrent Neural Networks](https://mlsanity.com/paper/1911.01952/) (2019)
- [Probabilistic Model Checking of Robots Deployed in Extreme Environments](https://mlsanity.com/paper/1812.04128/) (2018)
- [Assessing Safety-Critical Systems from Operational Testing: A Study on Autonomous Vehicles](https://mlsanity.com/paper/2008.09510/) (2020)
- [Operation is the hardest teacher: estimating DNN accuracy looking for mispredictions](https://mlsanity.com/paper/2102.04287/) (2021)
- [Detecting Operational Adversarial Examples for Reliable Deep Learning](https://mlsanity.com/paper/2104.06015/) (2021)
- [Finding the Mode of a Kernel Density Estimate](https://mlsanity.com/paper/1912.07673/) (2019)

## Built on by (most cited first, 11 of 11)

- [SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretability](https://mlsanity.com/paper/2208.09418/) (2022)
- [Reliability Assessment and Safety Arguments for Machine Learning Components in System Assurance](https://mlsanity.com/paper/2112.00646/) (2021)
- [Hierarchical Distribution-Aware Testing of Deep Learning](https://mlsanity.com/paper/2205.08589/) (2022)
- [What, Indeed, is an Achievable Provable Guarantee for Learning-Enabled Safety Critical Systems](https://mlsanity.com/paper/2307.11784/) (2023)
- [Probabilistic Robustness in Deep Learning: A Concise yet Comprehensive Guide](https://mlsanity.com/paper/2502.14833/) (2025)
- [Adversarial Robustness of Deep Learning: Theory, Algorithms, and Applications](https://mlsanity.com/paper/2108.10451/) (2021)
- [Keep your Distance: Determining Sampling and Distance Thresholds in Machine Learning Monitoring](https://mlsanity.com/paper/2207.05078/) (2022)
- [Reachability Verification Based Reliability Assessment for Deep Reinforcement Learning Controlled Robotics and Autonomous Systems](https://mlsanity.com/paper/2210.14991/) (2022)
- [Simulation-based Safety Assurance for an AVP System incorporating Learning-Enabled Components](https://mlsanity.com/paper/2311.03362/) (2023)
- [A Hierarchical Imprecise Probability Approach to Reliability Assessment of Large Language Models](https://mlsanity.com/paper/2511.00527/) (2025)
- [PRBench: A Standardized Probabilistic Robustness Benchmark](https://mlsanity.com/paper/2511.01724/) (2025)

## Abstract (arXiv)

The utilisation of Deep Learning (DL) is advancing into increasingly more sophisticated applications. While it shows great potential to provide transformational capabilities, DL also raises new challenges regarding its reliability in critical functions. In this paper, we present a model-agnostic reliability assessment method for DL classifiers, based on evidence from robustness evaluation and the operational profile (OP) of a given application. We partition the input space into small cells and then "assemble" their robustness (to the ground truth) according to the OP, where estimators on the cells' robustness and OPs are provided. Reliability estimates in terms of the probability of misclassification per input (pmi) can be derived together with confidence levels. A prototype tool is demonstrated with simplified case studies. Model assumptions and extension to real-world applications are also discussed. While our model easily uncovers the inherent difficulties of assessing the DL dependability (e.g. lack of data with ground truth and scalability issues), we provide preliminary/compromised solutions to advance in this research direction.