mlsanityData to 14 Oct 2026

Legible Failures: Detecting and Repairing In-Context Binding Errors

arXiv 2609.11216 ↗First posted 10 Sep 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

A wrong answer does not show whether the model lacked the needed information or held it and failed to use it. On an entity-obligation binding task, a language model can emit an incorrect prompt-supplied binding while a linear probe can recover the correct one from its frozen hidden state. We measure how often this occurs across 16 public checkpoints, each evaluated with three seeds. We fit a probe on a training fold, select its layer on a validation fold, and report results on a disjoint test fold. On the trials each model gets wrong, probe accuracy exceeds the strict present-obligation baseline, 1/K = 0.125, by +0.196 (95% CI [+0.101, +0.296], bootstrapped over models). A query-entity counterfactual rules out token presence and recency. A score built from the sign of probe-output disagreement improves failure detection over the model's own confidence by +0.079 AUROC (95% CI [+0.036, +0.126]). Raw probe confidence gives no measurable improvement over model confidence. Steering the residual stream toward the probe-decoded binding, with no gold label, raises accuracy on all eight models tested by a mean of +0.168 (95% CI [+0.066, +0.280]). Where recent studies report that probe-detected errors are resistant to interventions, we find that in-context binding is a setting in which probes are actionable.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 12 of its 13 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

12 of 13 references.

References · 12

  1. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  2. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  3. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  4. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  5. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
  6. 2023How do Language Models Bind Entities in Context?cited by 108
  7. 2023Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?cited by 62
  8. 2025Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Contextcited by 26
  9. 2026Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representationscited by 17
  10. 2025Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulnesscited by 10
  11. 2025Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questionscited by 6
  12. 2026Detection Without Correction: A Robust Asymmetry in Activation-Based Hallucination Probingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Decodable In-Context State and Model Output Across Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Knowing-Saying Gap: When Probes See Errors that Confidence Misses
  2. 2026Entity Binding Failures in Tool-Augmented Agents
  3. 2026How Language Models Fail: Token-Level Signatures of Committed and Persistent Reasoning Failures
  4. 2026Binding Drift in Multi-Step Tool-Augmented Agents
  5. 2025CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models
  6. 2026Diagnosing and Repairing Factual Errors in RAG under Budget Constraints
  7. 2026The Calls are Coming from Inside the Model: Investigating Probe-based Detection of Tool-Calling Errors in LLMs
  8. 2025Can LLMs Detect Their Confabulations? Estimating Reliability in Uncertainty-Aware Language Models