# Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations

arXiv: https://arxiv.org/abs/2302.02187 · HTML: https://mlsanity.com/paper/2302.02187/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 79 references that are in the map.
- First posted on arXiv 4 Feb 2023.

## Builds on (most cited first, 27 of 27)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI](https://mlsanity.com/paper/2010.07487/) (2020)
- [On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection](https://mlsanity.com/paper/1811.07901/) (2018)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- ["Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans](https://mlsanity.com/paper/2001.05871/) (2020)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation](https://mlsanity.com/paper/2204.11788/) (2022)
- [Do Explanations make VQA Models more Predictable to a Human?](https://mlsanity.com/paper/1810.12366/) (2018)
- [Does Explainable Artificial Intelligence Improve Human Decision-Making?](https://mlsanity.com/paper/2006.11194/) (2020)
- [Forming Effective Human-AI Teams: Building Machine Learning Models that Complement the Capabilities of Multiple Experts](https://mlsanity.com/paper/2206.07948/) (2022)
- [Visual correspondence-based explanations improve AI robustness and human-AI team accuracy](https://mlsanity.com/paper/2208.00780/) (2022)
- [Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2204.06916/) (2022)
- [Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2209.11812/) (2022)
- [Explainable AI for Interpretable Credit Scoring](https://mlsanity.com/paper/2012.03749/) (2020)
- [On the Influence of Explainable AI on Automation Bias](https://mlsanity.com/paper/2204.08859/) (2022)
- [Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QA](https://mlsanity.com/paper/2012.15075/) (2020)
- [On the Effect of Information Asymmetry in Human-AI Teams](https://mlsanity.com/paper/2205.01467/) (2022)
- [Do you comply with AI? -- Personalized explanations of learning algorithms and their impact on employees' compliance behavior](https://mlsanity.com/paper/2002.08777/) (2020)
- [Trust and Reliance in XAI -- Distinguishing Between Attitudinal and Behavioral Measures](https://mlsanity.com/paper/2203.12318/) (2022)
- [An Empirical Evaluation of Predicted Outcomes as Explanations in Human-AI Decision-Making](https://mlsanity.com/paper/2208.04181/) (2022)
- [Explaining Explanations to Society](https://mlsanity.com/paper/1901.06560/) (2019)
- [Training Novices: The Role of Human-AI Collaboration and Knowledge Transfer](https://mlsanity.com/paper/2207.00497/) (2022)

## Built on by (most cited first, 30 of 83)

- ["I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust](https://mlsanity.com/paper/2405.00623/) (2024)
- [Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant](https://mlsanity.com/paper/2502.01390/) (2025)
- [In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Making](https://mlsanity.com/paper/2305.07722/) (2023)
- [Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making](https://mlsanity.com/paper/2403.16812/) (2024)
- [A Decision Theoretic Framework for Measuring AI Reliance](https://mlsanity.com/paper/2401.15356/) (2024)
- [Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant](https://mlsanity.com/paper/2501.17546/) (2025)
- [The Impact of Imperfect XAI on Human-AI Decision-Making](https://mlsanity.com/paper/2307.13566/) (2023)
- [A Systematic Review on Fostering Appropriate Trust in Human-AI Interaction](https://mlsanity.com/paper/2311.06305/) (2023)
- [Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic review](https://mlsanity.com/paper/2310.19778/) (2023)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [A Critical Survey on Fairness Benefits of Explainable AI](https://mlsanity.com/paper/2310.13007/) (2023)
- [Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance](https://mlsanity.com/paper/2407.07950/) (2024)
- [Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making](https://mlsanity.com/paper/2401.07058/) (2024)
- [Contrastive Explanations That Anticipate Human Misconceptions Can Improve Human Decision-Making Skills](https://mlsanity.com/paper/2410.04253/) (2024)
- [AI, Help Me Thinkunicodex2014but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Support](https://mlsanity.com/paper/2504.06771/) (2025)
- [Measuring and mitigating overreliance to build human-compatible AI](https://mlsanity.com/paper/2509.08010/) (2025)
- [AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions](https://mlsanity.com/paper/2304.08804/) (2023)
- [On Evaluating Explanation Utility for Human-AI Decision Making in NLP](https://mlsanity.com/paper/2407.03545/) (2024)
- [From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis](https://mlsanity.com/paper/2502.11919/) (2025)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Towards Effective Human-AI Decision-Making: The Role of Human Learning in Appropriate Reliance on AI Advice](https://mlsanity.com/paper/2310.02108/) (2023)
- [Don't be Fooled: The Misinformation Effect of Explanations in Human-AI Collaboration](https://mlsanity.com/paper/2409.12809/) (2024)
- [Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance](https://mlsanity.com/paper/2502.13321/) (2025)
- [Motion2Meaning: A Clinician-Centered Framework for Contestable LLM in Parkinson's Disease Gait Interpretation](https://mlsanity.com/paper/2512.08934/) (2025)
- [Comparing Human Oversight Strategies for Computer-Use Agents](https://mlsanity.com/paper/2604.04918/) (2026)
- [Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents](https://mlsanity.com/paper/2606.05391/) (2026)
- [A Survey of AI Reliance](https://mlsanity.com/paper/2408.03948/) (2024)
- [Generative AI and Creativity: A Systematic Literature Review and Meta-Analysis](https://mlsanity.com/paper/2505.17241/) (2025)
- [Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translation](https://mlsanity.com/paper/2505.24683/) (2025)
- [Epistemology gives a Future to Complementarity in Human-AI Interactions](https://mlsanity.com/paper/2601.09871/) (2026)

## Abstract (arXiv)

AI advice is becoming increasingly popular, e.g., in investment and medical treatment decisions. As this advice is typically imperfect, decision-makers have to exert discretion as to whether actually follow that advice: they have to "appropriately" rely on correct and turn down incorrect advice. However, current research on appropriate reliance still lacks a common definition as well as an operational measurement concept. Additionally, no in-depth behavioral experiments have been conducted that help understand the factors influencing this behavior. In this paper, we propose Appropriateness of Reliance (AoR) as an underlying, quantifiable two-dimensional measurement concept. We develop a research model that analyzes the effect of providing explanations for AI advice. In an experiment with 200 participants, we demonstrate how these explanations influence the AoR, and, thus, the effectiveness of AI advice. Our work contributes fundamental concepts for the analysis of reliance behavior and the purposeful design of AI advisors.