# Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making

arXiv: https://arxiv.org/abs/2005.10960 · HTML: https://mlsanity.com/paper/2005.10960/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2020.

## Built on by (most cited first, 25 of 25)

- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [Improving Expert Predictions with Conformal Prediction](https://mlsanity.com/paper/2201.12006/) (2022)
- [Designing for Responsible Trust in AI Systems: A Communication Perspective](https://mlsanity.com/paper/2204.13828/) (2022)
- [Teaching Humans When To Defer to a Classifier via Exemplars](https://mlsanity.com/paper/2111.11297/) (2021)
- [Designing Decision Support Systems Using Counterfactual Prediction Sets](https://mlsanity.com/paper/2306.03928/) (2023)
- [Towards Human-AI Complementarity with Prediction Sets](https://mlsanity.com/paper/2405.17544/) (2024)
- [Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development](https://mlsanity.com/paper/2110.02932/) (2021)
- [Effective Human-AI Teams via Learned Natural Language Rules and Onboarding](https://mlsanity.com/paper/2311.01007/) (2023)
- [Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic review](https://mlsanity.com/paper/2310.19778/) (2023)
- [Intuitively Assessing ML Model Reliability through Example-Based Explanations and Editing Model Inputs](https://mlsanity.com/paper/2102.08540/) (2021)
- [Controlling Counterfactual Harm in Decision Support Systems Based on Prediction Sets](https://mlsanity.com/paper/2406.06671/) (2024)
- [Towards Human-AI Complementarity in Matching Tasks](https://mlsanity.com/paper/2508.13285/) (2025)
- [Human-AI Collaborative Uncertainty Quantification](https://mlsanity.com/paper/2510.23476/) (2025)
- [Human-Alignment Influences the Utility of AI-assisted Decision Making](https://mlsanity.com/paper/2501.14035/) (2025)
- [Painting the black box white: experimental findings from applying XAI to an ECG reading setting](https://mlsanity.com/paper/2210.15236/) (2022)
- [The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features](https://mlsanity.com/paper/2310.08617/) (2023)
- [Narrowing Action Choices with AI Improves Human Sequential Decisions](https://mlsanity.com/paper/2510.16097/) (2025)
- [Conformal Set-based Human-AI Complementarity with Multiple Experts](https://mlsanity.com/paper/2508.06997/) (2025)
- [Cybersecurity is the True Frontier for Generative AI Success or Failure](https://mlsanity.com/paper/2606.28929/) (2026)
- [A Mathematical Model of the Hidden Feedback Loop Effect in Machine Learning Systems](https://mlsanity.com/paper/2405.02726/) (2024)
- [Learning to Decide with AI Assistance under Human-Alignment](https://mlsanity.com/paper/2605.12646/) (2026)
- [Human Response to an AI-Based Decision Support System: A User Study on the Effects of Accuracy and Bias](https://mlsanity.com/paper/2203.15514/) (2022)
- [Exploring How Anomalous Model Input and Output Alerts Affect Decision-Making in Healthcare](https://mlsanity.com/paper/2204.13194/) (2022)
- [A Meta-heuristic Approach to Estimate and Explain Classifier Uncertainty](https://mlsanity.com/paper/2304.10284/) (2023)
- [Not All Trust is the Same: Effects of Decision Workflow and Explanations in Human-AI Decision Making](https://mlsanity.com/paper/2603.05229/) (2026)

## Abstract (arXiv)

ML decision-aid systems are increasingly common on the web, but their successful integration relies on people trusting them appropriately: they should use the system to fill in gaps in their ability, but recognize signals that the system might be incorrect. We measured how people's trust in ML recommendations differs by expertise and with more system information through a task-based study of 175 adults. We used two tasks that are difficult for humans: comparing large crowd sizes and identifying similar-looking animals. Our results provide three key insights: (1) People trust incorrect ML recommendations for tasks that they perform correctly the majority of the time, even if they have high prior knowledge about ML or are given information indicating the system is not confident in its prediction; (2) Four different types of system information all increased people's trust in recommendations; and (3) Math and logic skills may be as important as ML for decision-makers working with ML recommendations.