mlsanityData to 7 Oct 2026

Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making

arXiv 2005.10960 ↗First posted 22 May 2020cs.HC cs.AI cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

ML decision-aid systems are increasingly common on the web, but their successful integration relies on people trusting them appropriately: they should use the system to fill in gaps in their ability, but recognize signals that the system might be incorrect. We measured how people's trust in ML recommendations differs by expertise and with more system information through a task-based study of 175 adults. We used two tasks that are difficult for humans: comparing large crowd sizes and identifying similar-looking animals. Our results provide three key insights: (1) People trust incorrect ML recommendations for tasks that they perform correctly the majority of the time, even if they have high prior knowledge about ML or are given information indicating the system is not confident in its prediction; (2) Four different types of system information all increased people's trust in recommendations; and (3) Math and logic skills may be as important as ML for decision-makers working with ML recommendations.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 5; 2024: 3; 2025: 5; 2026: 3.

25 papers.

Sort:

  1. 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
  2. 2022Improving Expert Predictions with Conformal Predictioncited by 54
  3. 2022Designing for Responsible Trust in AI Systems: A Communication Perspectivecited by 48
  4. 2021Teaching Humans When To Defer to a Classifier via Exemplarscited by 29
  5. 2023Designing Decision Support Systems Using Counterfactual Prediction Setscited by 26
  6. 2024Towards Human-AI Complementarity with Prediction Setscited by 25
  7. 2021Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Developmentcited by 21
  8. 2023Effective Human-AI Teams via Learned Natural Language Rules and Onboardingcited by 20
  9. 2023Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic reviewcited by 18
  10. 2021Intuitively Assessing ML Model Reliability through Example-Based Explanations and Editing Model Inputscited by 9
  11. 2024Controlling Counterfactual Harm in Decision Support Systems Based on Prediction Setscited by 9
  12. 2025Towards Human-AI Complementarity in Matching Taskscited by 7
13 more
  1. 2025Human-AI Collaborative Uncertainty Quantificationcited by 6
  2. 2025Human-Alignment Influences the Utility of AI-assisted Decision Makingcited by 5
  3. 2022Painting the black box white: experimental findings from applying XAI to an ECG reading settingcited by 4
  4. 2023The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Featurescited by 3
  5. 2025Narrowing Action Choices with AI Improves Human Sequential Decisionscited by 3
  6. 2025Conformal Set-based Human-AI Complementarity with Multiple Expertscited by 2
  7. 2026Cybersecurity is the True Frontier for Generative AI Success or Failurecited by 2
  8. 2024A Mathematical Model of the Hidden Feedback Loop Effect in Machine Learning Systemscited by 1
  9. 2026Learning to Decide with AI Assistance under Human-Alignmentcited by 1
  10. 2022Human Response to an AI-Based Decision Support System: A User Study on the Effects of Accuracy and Biascited by 0
  11. 2022Exploring How Anomalous Model Input and Output Alerts Affect Decision-Making in Healthcarecited by 0
  12. 2023A Meta-heuristic Approach to Estimate and Explain Classifier Uncertaintycited by 0
  13. 2026Not All Trust is the Same: Effects of Decision Workflow and Explanations in Human-AI Decision Makingcited by 0