# Robustness quantification: a new method for assessing the reliability of the predictions of a classifier

arXiv: https://arxiv.org/abs/2503.22418 · HTML: https://mlsanity.com/paper/2503.22418/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 28 Mar 2025.

## Builds on (most cited first, 9 of 9)

- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Aleatoric and Epistemic Uncertainty with Random Forests](https://mlsanity.com/paper/2001.00893/) (2020)
- [Label-wise Aleatoric and Epistemic Uncertainty Quantification](https://mlsanity.com/paper/2406.02354/) (2024)
- [Towards Robust Classification with Deep Generative Forests](https://mlsanity.com/paper/2007.05721/) (2020)

## Built on by (most cited first, 4 of 4)

- [Robustness and uncertainty: two complementary aspects of the reliability of the predictions of a classifier](https://mlsanity.com/paper/2512.15492/) (2025)
- [Robustness Quantification and Uncertainty Quantification: Comparing Two Methods for Assessing the Reliability of Classifier Predictions](https://mlsanity.com/paper/2603.22988/) (2026)
- [Robustness Quantification for Discriminative Models: a New Robustness Metric and its Application to Dynamic Classifier Selection](https://mlsanity.com/paper/2603.23318/) (2026)
- [Local Robustness Quantification for Naive Bayes Classifiers and Generative Forests: a General Approach](https://mlsanity.com/paper/2609.11366/) (2026)

## Abstract (arXiv)

Based on existing ideas in the field of imprecise probabilities, we present a new approach for assessing the reliability of the individual predictions of a generative probabilistic classifier. We call this approach robustness quantification, compare it to uncertainty quantification, and demonstrate that it continues to work well even for classifiers that are learned from small training sets that are sampled from a shifted distribution.