# A Decision Theoretic Framework for Measuring AI Reliance

arXiv: https://arxiv.org/abs/2401.15356 · HTML: https://mlsanity.com/paper/2401.15356/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 27 Jan 2024.

## Builds on (most cited first, 20 of 20)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [To Trust Or Not To Trust A Classifier](https://mlsanity.com/paper/1805.11783/) (2018)
- [Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?](https://mlsanity.com/paper/2005.01831/) (2020)
- [On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection](https://mlsanity.com/paper/1811.07901/) (2018)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- [Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations](https://mlsanity.com/paper/2302.02187/) (2023)
- [Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations](https://mlsanity.com/paper/2301.07255/) (2023)
- [Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making](https://mlsanity.com/paper/2301.05809/) (2023)
- [Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making](https://mlsanity.com/paper/2101.05303/) (2021)
- [What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play](https://mlsanity.com/paper/1810.09648/) (2018)
- [In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Making](https://mlsanity.com/paper/2305.07722/) (2023)
- [Use-Case-Grounded Simulations for Explanation Evaluation](https://mlsanity.com/paper/2206.02256/) (2022)
- [The Impact of Algorithmic Risk Assessments on Human Predictions and its Analysis via Crowdsourcing Studies](https://mlsanity.com/paper/2109.01443/) (2021)
- [The Rational Agent Benchmark for Data Visualization](https://mlsanity.com/paper/2304.03432/) (2023)
- [Rating Reliability and Bias in News Articles: Does AI Assistance Help Everyone?](https://mlsanity.com/paper/1904.01531/) (2019)

## Built on by (most cited first, 28 of 28)

- [The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment](https://mlsanity.com/paper/2412.16468/) (2024)
- [Measuring and mitigating overreliance to build human-compatible AI](https://mlsanity.com/paper/2509.08010/) (2025)
- [Conformal Prediction and Human Decision Making](https://mlsanity.com/paper/2503.11709/) (2025)
- [AI Agents for Inventory Control: Human-LLM-OR Complementarity](https://mlsanity.com/paper/2602.12631/) (2026)
- [When Should We Orchestrate Multiple Agents?](https://mlsanity.com/paper/2503.13577/) (2025)
- [Human-AI Collaborative Uncertainty Quantification](https://mlsanity.com/paper/2510.23476/) (2025)
- [Explaining and Improving Information Complementarities in Multi-Agent Decision-making](https://mlsanity.com/paper/2502.06152/) (2025)
- [A Survey of AI Reliance](https://mlsanity.com/paper/2408.03948/) (2024)
- [Revisiting Rogers' Paradox in the Context of Human-AI Interaction](https://mlsanity.com/paper/2501.10476/) (2025)
- [How Performance Pressure Influences AI-Assisted Decision Making](https://mlsanity.com/paper/2410.16560/) (2024)
- [Unexploited Information Value in Human-AI Collaboration](https://mlsanity.com/paper/2411.10463/) (2024)
- [Explanations are a Means to an End: Decision Theoretic Explanation Evaluation](https://mlsanity.com/paper/2506.22740/) (2025)
- [Delegation and Verification Under AI](https://mlsanity.com/paper/2603.02961/) (2026)
- [From Accuracy to Readiness: Metrics and Benchmarks for Human-AI Decision-Making](https://mlsanity.com/paper/2603.18895/) (2026)
- [CollabSkill: Evaluating Human-Agent Collaboration On Real-World Tasks](https://mlsanity.com/paper/2606.09833/) (2026)
- [Unequal Uncertainty: Rethinking Algorithmic Interventions for Mitigating Discrimination from AI](https://mlsanity.com/paper/2508.07872/) (2025)
- [2-Step Agent: A Framework for the Interaction of a Decision Maker with AI Decision Support](https://mlsanity.com/paper/2602.21889/) (2026)
- [CoAX: Cognitive-Oriented Attribution eXplanation User Model of Human Understanding of AI Explanations](https://mlsanity.com/paper/2604.27354/) (2026)
- [RuleEdit: Failure-Guided Human-AI Model Editing with Prospective Impact Preview](https://mlsanity.com/paper/2606.00011/) (2026)
- [Human-AI Teaming Through the Lens of Calibration](https://mlsanity.com/paper/2606.10906/) (2026)
- [Human Decision-Making with AI Assistance under Correlated Features](https://mlsanity.com/paper/2606.20628/) (2026)
- [Robust Human-AI Complementarity under Uncertainty](https://mlsanity.com/paper/2607.06656/) (2026)
- [Characterizing Rhetorical Misalignment in Decision-Making with Language Models](https://mlsanity.com/paper/2608.14630/) (2026)
- [One AI Signal, Many Human Judgments: A Bayesian Cascade Analysis of AI-based Credibility Indicators in Online Information Spread](https://mlsanity.com/paper/2608.30311/) (2026)
- [Authority-Preserving Evaluation of Medical Vision-Language Assistants](https://mlsanity.com/paper/2609.22302/) (2026)
- [Annie, Are You Okay? How Style- and Context-Based Personalization Shape AI-Assisted Decision-Making](https://mlsanity.com/paper/2609.24644/) (2026)
- [Bounded Reasoning: Cognitive Hierarchy in Human-versus-AI Cyber Defense](https://mlsanity.com/paper/2610.04878/) (2026)
- [Trust Hinders Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2610.15136/) (2026)

## Abstract (arXiv)

Humans frequently make decisions with the aid of artificially intelligent (AI) systems. A common pattern is for the AI to recommend an action to the human who retains control over the final decision. Researchers have identified ensuring that a human has appropriate reliance on an AI as a critical component of achieving complementary performance. We argue that the current definition of appropriate reliance used in such research lacks formal statistical grounding and can lead to contradictions. We propose a formal definition of reliance, based on statistical decision theory, which separates the concepts of reliance as the probability the decision-maker follows the AI's recommendation from challenges a human may face in differentiating the signals and forming accurate beliefs about the situation. Our definition gives rise to a framework that can be used to guide the design and interpretation of studies on human-AI complementarity and reliance. Using recent AI-advised decision making studies from literature, we demonstrate how our framework can be used to separate the loss due to mis-reliance from the loss due to not accurately differentiating the signals. We evaluate these losses by comparing to a baseline and a benchmark for complementary performance defined by the expected payoff achieved by a rational decision-maker facing the same decision task as the behavioral decision-makers.