# U-Trustworthy Models.Reliability, Competence, and Confidence in Decision-Making

arXiv: https://arxiv.org/abs/2401.02062 · HTML: https://mlsanity.com/paper/2401.02062/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 51 references that are in the map.
- First posted on arXiv 4 Jan 2024.

## Builds on (most cited first, 20 of 20)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [To Trust Or Not To Trust A Classifier](https://mlsanity.com/paper/1805.11783/) (2018)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Fairness in Machine Learning](https://mlsanity.com/paper/2012.15816/) (2020)
- [Algorithmic Fairness](https://mlsanity.com/paper/2001.09784/) (2020)
- [AUC Maximization in the Era of Big Data and AI: A Survey](https://mlsanity.com/paper/2203.15046/) (2022)
- [Calibration tests in multi-class classification: A unifying framework](https://mlsanity.com/paper/1910.11385/) (2019)
- [The relationship between trust in AI and trustworthy machine learning technologies](https://mlsanity.com/paper/1912.00782/) (2019)
- [On the Consistency of Top-k Surrogate Losses](https://mlsanity.com/paper/1901.11141/) (2019)
- [Towards Trustworthy Predictions from Deep Neural Networks with Fast Adversarial Calibration](https://mlsanity.com/paper/2012.10923/) (2020)
- [The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations](https://mlsanity.com/paper/2212.11737/) (2022)
- [How Much Can We Really Trust You? Towards Simple, Interpretable Trust Quantification Metrics for Deep Neural Networks](https://mlsanity.com/paper/2009.05835/) (2020)
- [Optimizing Two-way Partial AUC with an End-to-end Framework](https://mlsanity.com/paper/2206.11655/) (2022)
- [Learning to Predict Trustworthiness with Steep Slope Loss](https://mlsanity.com/paper/2110.00054/) (2021)
- [Practices for Engineering Trustworthy Machine Learning Applications](https://mlsanity.com/paper/2103.00964/) (2021)
- [Toward a Taxonomy of Trust for Probabilistic Machine Learning](https://mlsanity.com/paper/2112.03270/) (2021)
- [The Value of AI Guidance in Human Examination of Synthetically-Generated Faces](https://mlsanity.com/paper/2208.10544/) (2022)
- [Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability](https://mlsanity.com/paper/2209.13262/) (2022)

## Built on by (most cited first, 1 of 1)

- [I-trustworthy Models. A framework for trustworthiness evaluation of probabilistic classifiers](https://mlsanity.com/paper/2501.15617/) (2025)

## Abstract (arXiv)

With growing concerns regarding bias and discrimination in predictive models, the AI community has increasingly focused on assessing AI system trustworthiness. Conventionally, trustworthy AI literature relies on the probabilistic framework and calibration as prerequisites for trustworthiness. In this work, we depart from this viewpoint by proposing a novel trust framework inspired by the philosophy literature on trust. We present a precise mathematical definition of trustworthiness, termed U-trustworthiness, specifically tailored for a subset of tasks aimed at maximizing a utility function. We argue that a model's U-trustworthiness is contingent upon its ability to maximize Bayes utility within this task subset. Our first set of results challenges the probabilistic framework by demonstrating its potential to favor less trustworthy models and introduce the risk of misleading trustworthiness assessments. Within the context of U-trustworthiness, we prove that properly-ranked models are inherently U-trustworthy. Furthermore, we advocate for the adoption of the AUC metric as the preferred measure of trustworthiness. By offering both theoretical guarantees and experimental validation, AUC enables robust evaluation of trustworthiness, thereby enhancing model selection and hyperparameter tuning to yield more trustworthy outcomes.