# Legible Failures: Detecting and Repairing In-Context Binding Errors

arXiv: https://arxiv.org/abs/2609.11216 · HTML: https://mlsanity.com/paper/2609.11216/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 13 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs](https://mlsanity.com/paper/2406.15927/) (2024)
- [LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations](https://mlsanity.com/paper/2410.02707/) (2024)
- [How do Language Models Bind Entities in Context?](https://mlsanity.com/paper/2310.17191/) (2023)
- [Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?](https://mlsanity.com/paper/2312.03729/) (2023)
- [Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context](https://mlsanity.com/paper/2510.06182/) (2025)
- [Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations](https://mlsanity.com/paper/2603.18353/) (2026)
- [Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulness](https://mlsanity.com/paper/2510.09033/) (2025)
- [Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questions](https://mlsanity.com/paper/2509.23782/) (2025)
- [Detection Without Correction: A Robust Asymmetry in Activation-Based Hallucination Probing](https://mlsanity.com/paper/2604.13068/) (2026)

## Built on by (most cited first, 1 of 1)

- [Decodable In-Context State and Model Output Across Training](https://mlsanity.com/paper/2609.31401/) (2026)

## Abstract (arXiv)

A wrong answer does not show whether the model lacked the needed information or held it and failed to use it. On an entity-obligation binding task, a language model can emit an incorrect prompt-supplied binding while a linear probe can recover the correct one from its frozen hidden state. We measure how often this occurs across 16 public checkpoints, each evaluated with three seeds. We fit a probe on a training fold, select its layer on a validation fold, and report results on a disjoint test fold. On the trials each model gets wrong, probe accuracy exceeds the strict present-obligation baseline, 1/K = 0.125, by +0.196 (95% CI [+0.101, +0.296], bootstrapped over models). A query-entity counterfactual rules out token presence and recency. A score built from the sign of probe-output disagreement improves failure detection over the model's own confidence by +0.079 AUROC (95% CI [+0.036, +0.126]). Raw probe confidence gives no measurable improvement over model confidence. Steering the residual stream toward the probe-decoded binding, with no gold label, raises accuracy on all eight models tested by a mean of +0.168 (95% CI [+0.066, +0.280]). Where recent studies report that probe-detected errors are resistant to interventions, we find that in-context binding is a setting in which probes are actionable.