# Justification-Based Reliability in Machine Learning

arXiv: https://arxiv.org/abs/1911.07391 · HTML: https://mlsanity.com/paper/1911.07391/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2019.

## Built on by (most cited first, 5 of 5)

- [A Survey on Uncertainty Reasoning and Quantification for Decision Making: Belief Theory Meets Deep Learning](https://mlsanity.com/paper/2206.05675/) (2022)
- [Extracting Semantics from Maintenance Records](https://mlsanity.com/paper/2108.05454/) (2021)
- [Competency-Aware Planning for Probabilistically Safe Navigation Under Perception Uncertainty](https://mlsanity.com/paper/2409.06111/) (2024)
- [Do-AIQ: A Design-of-Experiment Approach to Quality Evaluation of AI Mislabel Detection Algorithm](https://mlsanity.com/paper/2208.09953/) (2022)
- [Extracting Explanations, Justification, and Uncertainty from Black-Box Deep Neural Networks](https://mlsanity.com/paper/2403.08652/) (2024)

## Abstract (arXiv)

With the advent of Deep Learning, the field of machine learning (ML) has surpassed human-level performance on diverse classification tasks. At the same time, there is a stark need to characterize and quantify reliability of a model's prediction on individual samples. This is especially true in application of such models in safety-critical domains of industrial control and healthcare. To address this need, we link the question of reliability of a model's individual prediction to the epistemic uncertainty of the model's prediction. More specifically, we extend the theory of Justified True Belief (JTB) in epistemology, created to study the validity and limits of human-acquired knowledge, towards characterizing the validity and limits of knowledge in supervised classifiers. We present an analysis of neural network classifiers linking the reliability of its prediction on an input to characteristics of the support gathered from the input and latent spaces of the network. We hypothesize that the JTB analysis exposes the epistemic uncertainty (or ignorance) of a model with respect to its inference, thereby allowing for the inference to be only as strong as the justification permits. We explore various forms of support (for e.g., k-nearest neighbors (k-NN) and l_p-norm based) generated for an input, using the training data to construct a justification for the prediction with that input. Through experiments conducted on simulated and real datasets, we demonstrate that our approach can provide reliability for individual predictions and characterize regions where such reliability cannot be ascertained.