Perfect Detection, Failed Control: The Geometry of Knowing vs. Steering in Language Models
On the map: Mechanistic interpretability → Probing deception, bias and beliefs
Abstract (arXiv)
A central aspiration of mechanistic interpretability is controllability: if we know where a behavior is represented in a model's activations, we should be able to modify it. This rests on a hidden premise -- that the direction which detects a behavior and the direction which controls it are the same, or close. We test this geometrically: what is the angle between the direction that best detects a behavior and the one that best causes it? If detection implies control the cosine is near 1; otherwise it quantifies a detection-intervention gap. On Gemma 2-2B-it, output format (clean JSON vs markdown fencing) collapses both roles onto one axis. Hallucination does not: the model detects fake entities with perfect linear separability (AUC = 1.000 from layer 5), yet that direction sits at cos = 0.12 (about 83 degrees) from the direction producing a refusal -- a small, reproducible alignment, far from the cos = 1 that "detection is control" would require. A detector built from activations, with no chosen tokens, likewise fails to align (cos = -0.06). The gap generalizes: across four models from three families and two scales (1B-9B), cos stays in [0.12, 0.20], identical before and after instruction tuning (0.1197 vs 0.1200), placing its origin in pretraining. A 15-degree rotation toward the refusal direction partially bridges it -- 73% and 60% refusal on two held-out fake-entity categories at 1.8% false positives. We then ask whether this cosine predicts steerability, and it does not: detection is a high-dimensional class, not a single direction, and what separates the steerable case is functional, not readable from a static angle. The cosine is a weight-computable signature of the dissociation between knowing and steering, not a predictor of it.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability. Builds on 0 of its 3 references that are in the map. First posted on arXiv 23 Jun 2026.
Builds on
0 of 3 references.
No references here yet.
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Asymmetries in Spontaneous and Instructed Deceptioncited by 0
- 2026Locating and Steering Refusal Beyond Attentioncited by 0
- 2026Steering Interference Reflects the Model's Defaults, Not the Behavior Directionscited by 0
- 2026Capacity, Responsiveness and Alignment: What Makes a Latent Structure Actionablecited by 0
- 2026Does the Model Use the Feature? Separating Steering from Mechanism in LLMscited by 0
- 2026Does the Model Use the Feature? Separating Steering from Mechanism in LLMs5 Oct 2026
- 2026Capacity, Responsiveness and Alignment: What Makes a Latent Structure Actionable28 Sep 2026
- 2026Steering Interference Reflects the Model's Defaults, Not the Behavior Directions7 Sep 2026
- 2026Locating and Steering Refusal Beyond Attention4 Sep 2026
- 2026Asymmetries in Spontaneous and Instructed Deception31 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Where Does Reasoning Break? Step-Level Hallucination Detection via Hidden-State Transport Geometry
- 2026Steering Safely or Off a Cliff? Rethinking Specificity and Robustness in Inference-Time Interventions
- 2026Curved Inference II: Sleeper Agent Geometry - Extending Interpretability Beyond Probes
- 2026From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data
- 2026A Geometric Account of Activation Steering through Angle-Norm Decomposition
- 2026Detection Without Correction: A Robust Asymmetry in Activation-Based Hallucination Probing
- 2026Measuring Activation Control in Large Language Models
- 2026The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models