mlsanityData to 14 Oct 2026

Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It

arXiv 2608.17809 ↗First posted 18 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Humans naturally form and express beliefs in daily communication, e.g., "I think the answer is 3" or "I suppose that's right." Such beliefs inevitably intertwine with fact and knowledge, making the ability to handle them in tandem desirable for large language models (LLMs), as they are increasingly deployed in user-facing settings. Prior work showed that even capable LLMs exhibit a systemic weakness in acknowledging user beliefs grounded in incorrect information. We extend this evaluation to 10 LLMs across 18 epistemic expressions and find that the size and direction of this weakness depend on the verb used to express the belief, with the accuracy gap between factual and false information ranging from +50% on "I vaguely remember" to -14% on "I seriously doubt". We further show that the phenomenon stems from what we call task confusion: models default to fact-checking the underlying claim, overriding the user's stated belief. We provide evidence where chains of thought that explicitly fact-check show lower accuracy on false information than those that do not, and a single instruction can reverse the failure across verb families. Mechanistically, models attend more to false beliefs they fail to confirm, but suppressing this attention at decoding time recovers accuracy only partially and only in some models, calling for future work on intervention methods. Our findings clarify prior results and show how fact-checking, a generally desirable behavior, can interfere with belief tracking in LLMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

16 of 23 references.

References · 16

  1. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  2. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  3. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  4. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  5. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  6. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  7. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 345
  8. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
  9. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
  10. 2023Do Large Language Models Know What They Don't Know?cited by 243
  11. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 222
  12. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
4 more
  1. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
  2. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  3. 2023Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Modelscited by 91
  4. 2022(QA)2: Question Answering with Questionable Assumptionscited by 51

Built on by

Not cited yet.