# Toward a Taxonomy of Trust for Probabilistic Machine Learning

arXiv: https://arxiv.org/abs/2112.03270 · HTML: https://mlsanity.com/paper/2112.03270/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 98 references that are in the map.
- First posted on arXiv 5 Dec 2021.

## Builds on (most cited first, 28 of 28)

- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [To Trust Or Not To Trust A Classifier](https://mlsanity.com/paper/1805.11783/) (2018)
- [Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI](https://mlsanity.com/paper/2010.07487/) (2020)
- [Explaining Explanations in AI](https://mlsanity.com/paper/1811.01439/) (2018)
- [Validating Bayesian Inference Algorithms with Simulation-Based Calibration](https://mlsanity.com/paper/1804.06788/) (2018)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [The relationship between trust in AI and trustworthy machine learning technologies](https://mlsanity.com/paper/1912.00782/) (2019)
- [When Machine Learning Meets Privacy: A Survey and Outlook](https://mlsanity.com/paper/2011.11819/) (2020)
- [Veridical Data Science](https://mlsanity.com/paper/1901.08152/) (2019)
- [Yes, but Did It Work?: Evaluating Variational Inference](https://mlsanity.com/paper/1802.02538/) (2018)
- [Covariances, Robustness, and Variational Bayes](https://mlsanity.com/paper/1709.02536/) (2017)
- [An Automatic Finite-Sample Robustness Metric: When Can Dropping a Little Data Make a Big Difference?](https://mlsanity.com/paper/2011.14999/) (2020)
- [Validated Variational Inference via Practical Posterior Error Bounds](https://mlsanity.com/paper/1910.04102/) (2019)
- [Visualization in Bayesian workflow](https://mlsanity.com/paper/1709.01449/) (2017)
- [Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayes](https://mlsanity.com/paper/1506.04088/) (2015)
- [The Geometry of Continuous Latent Space Models for Network Data](https://mlsanity.com/paper/1712.08641/) (2017)
- [Identifying the latent space geometry of network models through analysis of curvature](https://mlsanity.com/paper/2012.10559/) (2020)
- [Approximate Cross-Validation for Structured Models](https://mlsanity.com/paper/2006.12669/) (2020)
- [Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametrics](https://mlsanity.com/paper/2107.03584/) (2021)
- [Next Waves in Veridical Network Embedding](https://mlsanity.com/paper/2007.05385/) (2020)
- [Fast robustness quantification with variational Bayes](https://mlsanity.com/paper/1606.07153/) (2016)

## Built on by (most cited first, 8 of 8)

- [Interpretable Machine Learning for Discovery: Statistical Challenges & Opportunities](https://mlsanity.com/paper/2308.01475/) (2023)
- [RetailSynth: Synthetic Data Generation for Retail AI Systems Evaluation](https://mlsanity.com/paper/2312.14095/) (2023)
- [Verification and Validation for Trustworthy Scientific Machine Learning](https://mlsanity.com/paper/2502.15496/) (2025)
- [Ethical AI Governance: Methods for Evaluating Trustworthy AI](https://mlsanity.com/paper/2409.07473/) (2024)
- [SMRS: advocating a unified reporting standard for surrogate models in the artificial intelligence era](https://mlsanity.com/paper/2502.06753/) (2025)
- [Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds](https://mlsanity.com/paper/2506.21887/) (2025)
- [Fair Feature Importance Scores for Interpreting Tree-Based Methods and Surrogates](https://mlsanity.com/paper/2310.04352/) (2023)
- [U-Trustworthy Models.Reliability, Competence, and Confidence in Decision-Making](https://mlsanity.com/paper/2401.02062/) (2024)

## Abstract (arXiv)

Probabilistic machine learning increasingly informs critical decisions in medicine, economics, politics, and beyond. We need evidence to support that the resulting decisions are well-founded. To aid development of trust in these decisions, we develop a taxonomy delineating where trust in an analysis can break down: (1) in the translation of real-world goals to goals on a particular set of available training data, (2) in the translation of abstract goals on the training data to a concrete mathematical problem, (3) in the use of an algorithm to solve the stated mathematical problem, and (4) in the use of a particular code implementation of the chosen algorithm. We detail how trust can fail at each step and illustrate our taxonomy with two case studies: an analysis of the efficacy of microcredit and The Economist's predictions of the 2020 US presidential election. Finally, we describe a wide variety of methods that can be used to increase trust at each step of our taxonomy. The use of our taxonomy highlights steps where existing research work on trust tends to concentrate and also steps where establishing trust is particularly challenging.