mlsanityData to 14 Oct 2026

Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations

arXiv 2302.02187 ↗First posted 4 Feb 2023cs.AI cs.HCcited by 83

Not on the map. Why?

Abstract (arXiv)

AI advice is becoming increasingly popular, e.g., in investment and medical treatment decisions. As this advice is typically imperfect, decision-makers have to exert discretion as to whether actually follow that advice: they have to "appropriately" rely on correct and turn down incorrect advice. However, current research on appropriate reliance still lacks a common definition as well as an operational measurement concept. Additionally, no in-depth behavioral experiments have been conducted that help understand the factors influencing this behavior. In this paper, we propose Appropriateness of Reliance (AoR) as an underlying, quantifiable two-dimensional measurement concept. We develop a research model that analyzes the effect of providing explanations for AI advice. In an experiment with 200 participants, we demonstrate how these explanations influence the AoR, and, thus, the effectiveness of AI advice. Our work contributes fundamental concepts for the analysis of reliance behavior and the purposeful design of AI advisors.

Key facts. Cited by 83 papers in the map (data through 14 Oct 2026). Builds on 27 of its 79 references that are in the map. First posted on arXiv 4 Feb 2023.

Builds on

27 of 79 references.

References · 27

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
  3. 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 285
  4. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  5. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  6. 2016Model-Agnostic Interpretability of Machine Learningcited by 219
  7. 2020Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIcited by 167
  8. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  9. 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
  10. 2020"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humanscited by 86
  11. 2021Towards a Science of Human-AI Decision Making: A Survey of Empirical Studiescited by 86
  12. 2022Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderationcited by 67
15 more
  1. 2018Do Explanations make VQA Models more Predictable to a Human?cited by 52
  2. 2020Does Explainable Artificial Intelligence Improve Human Decision-Making?cited by 49
  3. 2022Forming Effective Human-AI Teams: Building Machine Learning Models that Complement the Capabilities of Multiple Expertscited by 40
  4. 2022Visual correspondence-based explanations improve AI robustness and human-AI team accuracycited by 39
  5. 2022Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Makingcited by 28
  6. 2022Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Makingcited by 25
  7. 2020Explainable AI for Interpretable Credit Scoringcited by 18
  8. 2022On the Influence of Explainable AI on Automation Biascited by 18
  9. 2020Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QAcited by 17
  10. 2022On the Effect of Information Asymmetry in Human-AI Teamscited by 17
  11. 2020Do you comply with AI? -- Personalized explanations of learning algorithms and their impact on employees' compliance behaviorcited by 11
  12. 2022Trust and Reliance in XAI -- Distinguishing Between Attitudinal and Behavioral Measurescited by 10
  13. 2022An Empirical Evaluation of Predicted Outcomes as Explanations in Human-AI Decision-Makingcited by 7
  14. 2019Explaining Explanations to Societycited by 6
  15. 2022Training Novices: The Role of Human-AI Collaboration and Knowledge Transfercited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 15; 2025: 23; 2026: 36.

83 papers.

Sort:

By area:

  1. 2024"I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trustcited by 67
  2. 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
  3. 2023In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Makingcited by 50
  4. 2024Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Makingcited by 38
  5. 2024A Decision Theoretic Framework for Measuring AI Reliancecited by 28
  6. 2025Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistantcited by 26
  7. 2023The Impact of Imperfect XAI on Human-AI Decision-Makingcited by 21
  8. 2023A Systematic Review on Fostering Appropriate Trust in Human-AI Interactioncited by 21
  9. 2023Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic reviewcited by 18
  10. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 18
  11. 2023A Critical Survey on Fairness Benefits of Explainable AIcited by 16
  12. 2024Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliancecited by 16
38 more
  1. 2024Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Makingcited by 14
  2. 2024Contrastive Explanations That Anticipate Human Misconceptions Can Improve Human Decision-Making Skillscited by 14
  3. 2025AI, Help Me Thinkunicodex2014but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Supportcited by 12
  4. 2025Measuring and mitigating overreliance to build human-compatible AIcited by 12
  5. 2023AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventionscited by 9
  6. 2024On Evaluating Explanation Utility for Human-AI Decision Making in NLPcited by 9
  7. 2025From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysiscited by 9
  8. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 9
  9. 2023Towards Effective Human-AI Decision-Making: The Role of Human Learning in Appropriate Reliance on AI Advicecited by 8
  10. 2024Don't be Fooled: The Misinformation Effect of Explanations in Human-AI Collaborationcited by 8
  11. 2025Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistancecited by 6
  12. 2025Motion2Meaning: A Clinician-Centered Framework for Contestable LLM in Parkinson's Disease Gait Interpretationcited by 6
  13. 2026Comparing Human Oversight Strategies for Computer-Use Agentscited by 6
  14. 2026Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agentscited by 6
  15. 2024A Survey of AI Reliancecited by 5
  16. 2025Generative AI and Creativity: A Systematic Literature Review and Meta-Analysiscited by 5
  17. 2025Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translationcited by 5
  18. 2026Epistemology gives a Future to Complementarity in Human-AI Interactionscited by 5
  19. 2023PCNN: Probable-Class Nearest-Neighbor Explanations Improve Fine-Grained Image Classification Accuracy for AIs and Humanscited by 3
  20. 2024Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Processcited by 3
  21. 2024Personalized Help for Optimizing Low-Skilled Users' Strategycited by 3
  22. 2025Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languagescited by 3
  23. 2025From Consumption to Collaboration: Measuring Interaction Patterns to Augment Human Cognition in Open-Ended Taskscited by 3
  24. 2025Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrestcited by 3
  25. 2025Human-AI Complementarity: A Goal for Amplified Oversightcited by 3
  26. 2023Explain To Decide: A Human-Centric Review on the Role of Explainable Artificial Intelligence in AI-assisted Decision Makingcited by 2
  27. 2025The Impact and Feasibility of Self-Confidence Shaping for AI-Assisted Decision-Makingcited by 2
  28. 2025When Bias Backfires: The Modulatory Role of Counterfactual Explanations on the Adoption of Algorithmic Bias in XAI-Supported Human Decision-Makingcited by 2
  29. 2025Autonomy by Design: Preserving Human Autonomy in AI Decision-Supportcited by 2
  30. 2026Investigating Writing Professionals' Relationships with Generative AI: How Combined Perceptions of Rivalry and Collaboration Shape Work Practices and Outcomescited by 2
  31. 2024Mapping the Potential of Explainable AI for Fairness Along the AI Lifecyclecited by 1
  32. 2024How Performance Pressure Influences AI-Assisted Decision Makingcited by 1
  33. 2025Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics Perspectivecited by 1
  34. 2025Bayesian Inference for Correlated Human Experts and Classifierscited by 1
  35. 2025CTG-Insight: A Multi-Agent Interpretable LLM Framework for Cardiotocography Analysis and Classificationcited by 1
  36. 2025Do Students Rely on AI? Analysis of Student-ChatGPT Conversations from a Field Studycited by 1
  37. 2026Align When They Want, Complement When They Need! Human-Centered Ensembles for Adaptive Human-AI Collaborationcited by 1
  38. 2026From Accuracy to Readiness: Metrics and Benchmarks for Human-AI Decision-Makingcited by 1

50 most cited of 83 All 83 (JSON)