# Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict

arXiv: https://arxiv.org/abs/2610.12360 · HTML: https://mlsanity.com/paper/2610.12360/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 49 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 23 of 23)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [OpenAI GPT-5 System Card](https://mlsanity.com/paper/2601.03267/) (2025)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [Measuring Progress on Scalable Oversight for Large Language Models](https://mlsanity.com/paper/2211.03540/) (2022)
- [AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions](https://mlsanity.com/paper/2506.09038/) (2025)
- [Know Your Limits: A Survey of Abstention in Large Language Models](https://mlsanity.com/paper/2407.18418/) (2024)
- [TRAIL: Trace Reasoning and Agentic Issue Localization](https://mlsanity.com/paper/2505.08638/) (2025)
- [ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM](https://mlsanity.com/paper/2408.12076/) (2024)
- [Retrieval-Augmented Generation with Conflicting Evidence](https://mlsanity.com/paper/2504.13079/) (2025)
- [Monitoring Monitorability](https://mlsanity.com/paper/2512.18311/) (2025)
- [ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration](https://mlsanity.com/paper/2511.21689/) (2025)
- [A Survey on the Honesty of Large Language Models](https://mlsanity.com/paper/2409.18786/) (2024)
- [Training LLMs for Honesty via Confessions](https://mlsanity.com/paper/2512.08093/) (2025)
- [Challenging the Evaluator: LLM Sycophancy Under User Rebuttal](https://mlsanity.com/paper/2509.16533/) (2025)
- [Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict](https://mlsanity.com/paper/2506.06485/) (2025)
- [Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback](https://mlsanity.com/paper/2506.11930/) (2025)
- [AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security](https://mlsanity.com/paper/2504.20965/) (2025)
- [How to Interpret Agent Behavior](https://mlsanity.com/paper/2605.13625/) (2026)
- [Measuring Epistemic Humility in Multimodal Large Language Models](https://mlsanity.com/paper/2509.09658/) (2025)

## Abstract (arXiv)

When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion? Existing evaluations of agentic systems focus primarily on task success, offering limited insight into how agents handle such conflicts. We propose to evaluate agents on epistemic humility (EH): the agent's willingness to recognize, act on, and communicate uncertainty during task execution. We operationalize EH through three trajectory-level behavioral dimensions: Identify, Solve, and Escalate (ISE). Through knowledge conflict, situations where the backbone language model's parametric knowledge contradicts the evidence it encounters, or where two contextual sources disagree, we evaluate two conflict settings: (1) controlled conflict and (2) naturally occurring conflict during multi-step agentic execution, each paired with matched no-conflict controls. Evaluating four agents, we find that higher task accuracy does not necessarily correspond to greater epistemic humility: some high-accuracy configurations recognize conflicts during execution but do not communicate unresolved uncertainty in their incorrect final answers. Trajectory-level analysis further reveals that agents frequently detect conflicts in early steps of execution but fail to maintain or resolve them in later steps. Finally, we show that model-level interventions can improve EH, but often at the cost of task accuracy, suggesting that epistemic humility emerges from the interaction among the backbone model, agent harness, and evaluation environment.