Coherent Without Grounding, Grounded Without Success: Observability and Epistemic Failure
Not on the map. Why?
Abstract (arXiv)
When an agent can articulate why something works, we typically take this as evidence of genuine understanding. This presupposes that effective action and correct explanation covary, and that coherent explanation reliably signals both. I argue that this assumption fails for contemporary Large Language Models (LLMs). I introduce what I call the Bidirectional Coherence Paradox: competence and grounding not only dissociate but invert across epistemic conditions. In low-observability domains, LLMs often act successfully while misidentifying the mechanisms that produce their success. In high-observability domains, they frequently generate explanations that accurately track observable causal structure yet fail to translate those diagnoses into effective intervention. In both cases, explanatory coherence remains intact, obscuring the underlying dissociation. Drawing on experiments in compiler optimization and hyperparameter tuning, I develop the Epistemic Triangle, a model of how priors, signals, and domain knowledge interact under varying observability. The results suggest that neither behavioral success nor explanatory accuracy alone suffices for attributing understanding. I argue that evaluating artificial epistemic agents requires a tripartite framework -- coherence, grounding, and a proper basing relation linking explanation to action. The systematic separation of knowing-that and knowing-how in LLMs thus challenges assumptions inherited from both epistemology and current AI evaluation practice.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 5 of its 30 references that are in the map. First posted on arXiv 30 Mar 2026.
Builds on
5 of 30 references.
References · 5
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
- 2016The Mythos of Model Interpretabilitycited by 716
- 2022Talking About Large Language Modelscited by 174
- 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
Built on by
Not cited yet.