# "Are You Really Sure?" Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Making

arXiv: https://arxiv.org/abs/2403.09552 · HTML: https://mlsanity.com/paper/2403.09552/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2024.

## Built on by (most cited first, 26 of 26)

- [Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant](https://mlsanity.com/paper/2502.01390/) (2025)
- [As Confidence Aligns: Exploring the Effect of AI Confidence on Human Self-confidence in Human-AI Decision Making](https://mlsanity.com/paper/2501.12868/) (2025)
- [Contrastive Explanations That Anticipate Human Misconceptions Can Improve Human Decision-Making Skills](https://mlsanity.com/paper/2410.04253/) (2024)
- [The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment](https://mlsanity.com/paper/2412.16468/) (2024)
- [Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance](https://mlsanity.com/paper/2502.13321/) (2025)
- [Less or More: Towards Glanceable Explanations for LLM Recommendations Using Ultra-Small Devices](https://mlsanity.com/paper/2502.19410/) (2025)
- [Human-AI Complementarity: A Goal for Amplified Oversight](https://mlsanity.com/paper/2510.26518/) (2025)
- [When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being](https://mlsanity.com/paper/2512.08937/) (2025)
- [The Impact and Feasibility of Self-Confidence Shaping for AI-Assisted Decision-Making](https://mlsanity.com/paper/2502.14311/) (2025)
- [Beyond Accuracy: How AI Metacognitive Sensitivity improves AI-assisted Decision Making](https://mlsanity.com/paper/2507.22365/) (2025)
- [Learning to Trust: How Humans Mentally Recalibrate AI Confidence Signals](https://mlsanity.com/paper/2603.22634/) (2026)
- [Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics Perspective](https://mlsanity.com/paper/2504.10918/) (2025)
- [Exploring the Impact of Explainable AI and Cognitive Capabilities on Users' Decisions](https://mlsanity.com/paper/2505.01192/) (2025)
- [Agentic AI with Orchestrator-Agent Trust: A Modular Visual Classification Framework with Trust-Aware Orchestration and RAG-Based Reasoning](https://mlsanity.com/paper/2507.10571/) (2025)
- ["Shall We Dig Deeper?": Designing and Evaluating Strategies for LLM Agents to Advance Knowledge Co-Construction in Asynchronous Online Discussions](https://mlsanity.com/paper/2509.23327/) (2025)
- [AI as Teammate or Tool? A Review of Human-AI Interaction in Decision Support](https://mlsanity.com/paper/2602.15865/) (2026)
- [More Isn't Always Better: Balancing Decision Accuracy and Conformity Pressures in Multi-AI Advice](https://mlsanity.com/paper/2603.22152/) (2026)
- [Development of Mental Models in Human-AI Collaboration: A Conceptual Framework](https://mlsanity.com/paper/2510.08104/) (2025)
- [When Life Gives You AI, Will You Turn It Into A Market for Lemons? Understanding How Information Asymmetries About AI System Capabilities Affect Market Outcomes and Adoption](https://mlsanity.com/paper/2601.21650/) (2026)
- [Human-AI Collaboration in Large Language Model-Integrated Building Energy Management Systems: The Role of User Domain Knowledge and AI Literacy](https://mlsanity.com/paper/2602.16140/) (2026)
- [Beyond the Steeper Curve: AI-Mediated Metacognitive Decoupling and the Limits of the Dunning-Kruger Metaphor](https://mlsanity.com/paper/2603.29681/) (2026)
- [Uncertainty Gating for Cost-Aware Explainable Artificial Intelligence](https://mlsanity.com/paper/2603.29915/) (2026)
- [Trust the AI, Doubt Yourself: The Effect of Urgency on Self-Confidence in Human-AI Interaction](https://mlsanity.com/paper/2604.07535/) (2026)
- [The Timing Dependencies of Trust: Speed, Accuracy, and cBCI Neuro-Decoupling in Human-AI Teams](https://mlsanity.com/paper/2605.25868/) (2026)
- [The Confidence Trap: Calibration Attacks for Graph Neural Networks](https://mlsanity.com/paper/2606.08467/) (2026)
- [Human-AI Coordination Zones: A Framework for Designing Human-in-the-Loop Experiences with Agentic AI](https://mlsanity.com/paper/2606.09848/) (2026)

## Abstract (arXiv)

In AI-assisted decision-making, it is crucial but challenging for humans to achieve appropriate reliance on AI. This paper approaches this problem from a human-centered perspective, "human self-confidence calibration". We begin by proposing an analytical framework to highlight the importance of calibrated human self-confidence. In our first study, we explore the relationship between human self-confidence appropriateness and reliance appropriateness. Then in our second study, We propose three calibration mechanisms and compare their effects on humans' self-confidence and user experience. Subsequently, our third study investigates the effects of self-confidence calibration on AI-assisted decision-making. Results show that calibrating human self-confidence enhances human-AI team performance and encourages more rational reliance on AI (in some aspects) compared to uncalibrated baselines. Finally, we discuss our main findings and provide implications for designing future AI-assisted decision-making interfaces.