# Human-Alignment Influences the Utility of AI-assisted Decision Making

arXiv: https://arxiv.org/abs/2501.14035 · HTML: https://mlsanity.com/paper/2501.14035/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 23 references that are in the map.
- First posted on arXiv 23 Jan 2025.

## Builds on (most cited first, 5 of 5)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [Human-Aligned Calibration for AI-Assisted Decision Making](https://mlsanity.com/paper/2306.00074/) (2023)
- [Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making](https://mlsanity.com/paper/2005.10960/) (2020)
- [Improving Screening Processes via Calibrated Subset Selection](https://mlsanity.com/paper/2202.01147/) (2022)

## Built on by (most cited first, 5 of 5)

- [From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered](https://mlsanity.com/paper/2506.07461/) (2025)
- [Narrowing Action Choices with AI Improves Human Sequential Decisions](https://mlsanity.com/paper/2510.16097/) (2025)
- [Multi-Round Human-AI Collaboration with User-Specified Requirements](https://mlsanity.com/paper/2602.17646/) (2026)
- [Learning to Decide with AI Assistance under Human-Alignment](https://mlsanity.com/paper/2605.12646/) (2026)
- [Strategic Decision Support for AI Agents](https://mlsanity.com/paper/2606.12587/) (2026)

## Abstract (arXiv)

Whenever an AI model is used to predict a relevant (binary) outcome in AI-assisted decision making, it is widely agreed that, together with each prediction, the model should provide an AI confidence value. However, it has been unclear why decision makers have often difficulties to develop a good sense on when to trust a prediction using AI confidence values. Very recently, Corvelo Benz and Gomez Rodriguez have argued that, for rational decision makers, the utility of AI-assisted decision making is inherently bounded by the degree of alignment between the AI confidence values and the decision maker's confidence on their own predictions. In this work, we empirically investigate to what extent the degree of alignment actually influences the utility of AI-assisted decision making. To this end, we design and run a large-scale human subject study (n=703) where participants solve a simple decision making task - an online card game - assisted by an AI model with a steerable degree of alignment. Our results show a positive association between the degree of alignment and the utility of AI-assisted decision making. In addition, our results also show that post-processing the AI confidence values to achieve multicalibration with respect to the participants' confidence on their own predictions increases both the degree of alignment and the utility of AI-assisted decision making.