# Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance

arXiv: https://arxiv.org/abs/2407.07950 · HTML: https://mlsanity.com/paper/2407.07950/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 68 references that are in the map.
- First posted on arXiv 10 Jul 2024.

## Builds on (most cited first, 28 of 28)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Calibration of Pre-trained Transformers](https://mlsanity.com/paper/2003.07892/) (2020)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [Reducing conversational agents' overconfidence through linguistic calibration](https://mlsanity.com/paper/2012.14983/) (2020)
- [Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty](https://mlsanity.com/paper/2401.06730/) (2024)
- [Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations](https://mlsanity.com/paper/2302.02187/) (2023)
- [Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations](https://mlsanity.com/paper/2301.07255/) (2023)
- [Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?](https://mlsanity.com/paper/2312.03729/) (2023)
- [Understanding User Experience in Large Language Model Interactions](https://mlsanity.com/paper/2401.08329/) (2024)
- [Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Models](https://mlsanity.com/paper/2405.06079/) (2024)
- [Mirages: On Anthropomorphism in Dialogue Systems](https://mlsanity.com/paper/2305.09800/) (2023)
- [Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2204.06916/) (2022)
- [AnthroScore: A Computational Linguistic Measure of Anthropomorphism](https://mlsanity.com/paper/2402.02056/) (2024)
- [Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2209.11812/) (2022)
- [A Diachronic Perspective on User Trust in AI under Uncertainty](https://mlsanity.com/paper/2310.13544/) (2023)
- [From "AI" to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust?](https://mlsanity.com/paper/2404.16047/) (2024)
- [Calibrating Structured Output Predictors for Natural Language Processing](https://mlsanity.com/paper/2004.04361/) (2020)
- [Alexa, Google, Siri: What are Your Pronouns? Gender and Anthropomorphism in the Design and Perception of Conversational Assistants](https://mlsanity.com/paper/2106.02578/) (2021)
- [Warmth and competence in human-agent cooperation](https://mlsanity.com/paper/2201.13448/) (2022)
- [Are language models rational? The case of coherence norms and belief revision](https://mlsanity.com/paper/2406.03442/) (2024)

## Built on by (most cited first, 16 of 16)

- [Sabotage Evaluations for Frontier Models](https://mlsanity.com/paper/2410.21514/) (2024)
- [MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs](https://mlsanity.com/paper/2505.24858/) (2025)
- [Thinking beyond the anthropomorphic paradigm benefits LLM research](https://mlsanity.com/paper/2502.09192/) (2025)
- [Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing](https://mlsanity.com/paper/2507.10587/) (2025)
- [HumT DumT: Measuring and controlling human-like language in LLMs](https://mlsanity.com/paper/2502.13259/) (2025)
- [Humanizing Machines: Rethinking LLM Anthropomorphism Through a Multi-Level Framework of Design](https://mlsanity.com/paper/2508.17573/) (2025)
- [From tools to thieves: Measuring and understanding public perceptions of AI through crowdsourced metaphors](https://mlsanity.com/paper/2501.18045/) (2025)
- [Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence](https://mlsanity.com/paper/2510.01395/) (2025)
- [Social Simulacra in the Wild: AI Agent Communities on Moltbook](https://mlsanity.com/paper/2603.16128/) (2026)
- [Personalized Help for Optimizing Low-Skilled Users' Strategy](https://mlsanity.com/paper/2411.09109/) (2024)
- [Humans overrely on overconfident language models, across languages](https://mlsanity.com/paper/2507.06306/) (2025)
- [Let Them Down Easy! Contextual Effects of LLM Guardrails on User Perceptions and Preferences](https://mlsanity.com/paper/2506.00195/) (2025)
- [Offloading Score: Measuring AI Reliance Through Counterfactual Workflows](https://mlsanity.com/paper/2605.29392/) (2026)
- [Voice "Cloning" is Style Transfer](https://mlsanity.com/paper/2605.16578/) (2026)
- [Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being](https://mlsanity.com/paper/2607.25057/) (2026)
- [TrustMI: Causally controlling how assistants trust their users](https://mlsanity.com/paper/2610.06064/) (2026)

## Abstract (arXiv)

The ability to communicate uncertainty, risk, and limitation is crucial for the safety of large language models. However, current evaluations of these abilities rely on simple calibration, asking whether the language generated by the model matches appropriate probabilities. Instead, evaluation of this aspect of LLM communication should focus on the behaviors of their human interlocutors: how much do they rely on what the LLM says? Here we introduce an interaction-centered evaluation framework called Rel-A.I. (pronounced "rely"}) that measures whether humans rely on LLM generations. We use this framework to study how reliance is affected by contextual features of the interaction (e.g, the knowledge domain that is being discussed), or the use of greetings communicating warmth or competence (e.g., "I'm happy to help!"). We find that contextual characteristics significantly affect human reliance behavior. For example, people rely 10% more on LMs when responding to questions involving calculations and rely 30% more on LMs that are perceived as more competent. Our results show that calibration and language quality alone are insufficient in evaluating the risks of human-LM interactions, and illustrate the need to consider features of the interactional context.