# StaDRe and StaDRo: Reliability and Robustness Estimation of ML-based Forecasting using Statistical Distance Measures

arXiv: https://arxiv.org/abs/2206.11116 · HTML: https://mlsanity.com/paper/2206.11116/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 34 references that are in the map.
- First posted on arXiv 17 Jun 2022.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [BREEDS: Benchmarks for Subpopulation Shift](https://mlsanity.com/paper/2008.04859/) (2020)
- [Learning by Transduction](https://mlsanity.com/paper/1301.7375/) (2013)
- [The Cramer Distance as a Solution to Biased Wasserstein Gradients](https://mlsanity.com/paper/1705.10743/) (2017)
- [SafeML: Safety Monitoring of Machine Learning Classifiers through Statistical Difference Measure](https://mlsanity.com/paper/2005.13166/) (2020)
- [Clustering time series under the Fr'echet distance](https://mlsanity.com/paper/1512.04349/) (2015)

## Built on by (most cited first, 3 of 3)

- [Scope Compliance Uncertainty Estimate](https://mlsanity.com/paper/2312.10801/) (2023)
- [Q-SafeML: Safety Assessment of Quantum Machine Learning via Quantum Distance Metrics](https://mlsanity.com/paper/2509.04536/) (2025)
- [Safer Skin Lesion Classification with Global Class Activation Probability Map Evaluation and SafeML](https://mlsanity.com/paper/2508.20776/) (2025)

## Abstract (arXiv)

Reliability estimation of Machine Learning (ML) models is becoming a crucial subject. This is particularly the case when such mboxmodels are deployed in safety-critical applications, as the decisions based on model predictions can result in hazardous situations. In this regard, recent research has proposed methods to achieve safe, mboxdependable, and reliable ML systems. One such method consists of detecting and analyzing distributional shift, and then measuring how such systems respond to these shifts. This was proposed in earlier work in SafeML. This work focuses on the use of SafeML for time series data, and on reliability and robustness estimation of ML-forecasting methods using statistical distance measures. To this end, distance measures based on the Empirical Cumulative Distribution Function (ECDF) proposed in SafeML are explored to measure Statistical-Distance Dissimilarity (SDD) across time series. We then propose SDD-based Reliability Estimate (StaDRe) and SDD-based Robustness (StaDRo) measures. With the help of a clustering technique, the similarity between the statistical properties of data seen during training and the forecasts is identified. The proposed method is capable of providing a link between dataset SDD and Key Performance Indicators (KPIs) of the ML models.