# Scope Compliance Uncertainty Estimate

arXiv: https://arxiv.org/abs/2312.10801 · HTML: https://mlsanity.com/paper/2312.10801/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 17 Dec 2023.

## Builds on (most cited first, 10 of 10)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Importance-Driven Deep Learning System Testing](https://mlsanity.com/paper/2002.03433/) (2020)
- [SafeML: Safety Monitoring of Machine Learning Classifiers through Statistical Difference Measure](https://mlsanity.com/paper/2005.13166/) (2020)
- [Efficient Certification of Spatial Robustness](https://mlsanity.com/paper/2009.09318/) (2020)
- [Artificial Intelligence Safety and Cybersecurity: a Timeline of AI Failures](https://mlsanity.com/paper/1610.07997/) (2016)
- [A Provable Defense for Deep Residual Networks](https://mlsanity.com/paper/1903.12519/) (2019)
- [Detection and Mitigation of Rare Subclasses in Deep Neural Network Classifiers](https://mlsanity.com/paper/1911.12780/) (2019)
- [Keep your Distance: Determining Sampling and Distance Thresholds in Machine Learning Monitoring](https://mlsanity.com/paper/2207.05078/) (2022)
- [StaDRe and StaDRo: Reliability and Robustness Estimation of ML-based Forecasting using Statistical Distance Measures](https://mlsanity.com/paper/2206.11116/) (2022)
- [A New ECDF Two-Sample Test Statistic](https://mlsanity.com/paper/2007.01360/) (2020)

## Built on by (most cited first, 3 of 3)

- [Q-SafeML: Safety Assessment of Quantum Machine Learning via Quantum Distance Metrics](https://mlsanity.com/paper/2509.04536/) (2025)
- [Incorporating Failure of Machine Learning in Dynamic Probabilistic Safety Assurance](https://mlsanity.com/paper/2506.06868/) (2025)
- [Safer Skin Lesion Classification with Global Class Activation Probability Map Evaluation and SafeML](https://mlsanity.com/paper/2508.20776/) (2025)

## Abstract (arXiv)

The zeitgeist of the digital era has been dominated by an expanding integration of Artificial Intelligence~(AI) in a plethora of applications across various domains. With this expansion, however, questions of the safety and reliability of these methods come have become more relevant than ever. Consequently, a run-time ML model safety system has been developed to ensure the model's operation within the intended context, especially in applications whose environments are greatly variable such as Autonomous Vehicles~(AVs). SafeML is a model-agnostic approach for performing such monitoring, using distance measures based on statistical testing of the training and operational datasets; comparing them to a predetermined threshold, returning a binary value whether the model should be trusted in the context of the observed data or be deemed unreliable. Although a systematic framework exists for this approach, its performance is hindered by: (1) a dependency on a number of design parameters that directly affect the selection of a safety threshold and therefore likely affect its robustness, (2) an inherent assumption of certain distributions for the training and operational sets, as well as (3) a high computational complexity for relatively large sets. This work addresses these limitations by changing the binary decision to a continuous metric. Furthermore, all data distribution assumptions are made obsolete by implementing non-parametric approaches, and the computational speed increased by introducing a new distance measure based on the Empirical Characteristics Functions~(ECF).