mlsanityData to 14 Oct 2026

Decodability is Not Causality: Dissociating Probe Readouts from Behavioral Drivers via SAE Decomposition

arXiv 2609.18080 ↗First posted 16 Sep 2026cs.AIcited by 2

On the map: Mechanistic interpretability → Probing deception, bias and beliefs

Abstract (arXiv)

Linear probes can decode safety-relevant concepts such as truthfulness from language-model activations, but probe accuracy may show only decodability, not that the features the probe weights causally drive model behavior. We demonstrate that this gap cannot be closed from the geometry of probe weights alone: the features geometrically aligned with probe direction need not be the ones the model uses, so causal relevance requires intervention. We introduce a feature-level diagnostic that decomposes a deployed True/False probe into sparse-autoencoder (SAE) features, ranks those features by both probe alignment and by gradient sensitivity of the model's behavior, and ablates the resulting shared, probe-only, and random feature sets under a coherence gate. On the truth probe of Buerger et al. (2024) (TTPD), applied in the instructed truth/deception setting of Long et al. (2025) for Gemma2-9B-Instruct, the two rankings overlap only weakly (about 12%, Spearman rho = 0.10), and ablation dissociates them sharply: features the probe shares with the model flip the output far more (up to 27%) than equally sized probe-only (6%) or random (1%) features at full coherence, while probe-only features instead perturb the probe's own readout. The dissociation holds across five seeds and a held-out split, and an activation-aware selection of features flips behavior nearly three times as often as the probe's geometric top features (17.6% vs. 6.1%). In this setting, therefore, the geometric projection of a probe's weight vector alone does not identify the features the model causally uses; however, combining probe information with feature activation statistics recovers substantially more behaviorally causal features, and coherence-gated SAE intervention is needed to separate them from probe readouts.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability. Builds on 20 of its 22 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

20 of 22 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States1 Jun
  2. 2025When Truthful Representations Flip Under Deceptive Instructions?29 Jul

Other references · 18

  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  2. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  3. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  4. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  5. 2019Designing and Interpreting Probes with Control Taskscited by 640
  6. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  7. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  8. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 124
  9. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 115
  10. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 86
  11. 2024Truth is Universal: Robust Detection of Lies in LLMscited by 79
  12. 2021Do Input Gradients Highlight Discriminative Features?cited by 52
6 more
  1. 2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43
  2. 2024Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in GPT-2 Smallcited by 34
  3. 2024Optimal ablation for interpretabilitycited by 25
  4. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?cited by 12
  5. 2026The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probescited by 12
  6. 2024How Reliable are Causal Probing Interventions?cited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026What Would Falsify It? A Variable Specific Evidence Standard for Mechanistic Claims About Self Explanationcited by 0
  2. 2026What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Linear Probe Accuracy Scales with Model Size and Benefits from Multi-Layer Ensembling
  2. 2026When Decodability Is Not Enough: Logical Validity Representations, Behavioral Dissociation, and Causal Tests in Language Models
  3. 2026Probe Generalization as Subspace Selection for OOD Deception Detection
  4. 2026One Probe Won't Catch Them All: Towards Targeted Deception Detection
  5. 2026Caught in the Act: Probes Effectively Detect Sabotage and Catch Unverbalized Deception
  6. 2025Microsaccade-Inspired Probing: Positional Encoding Perturbations Reveal LLM Misbehaviours
  7. 2026Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations
  8. 2026The Knowing-Saying Gap: When Probes See Errors that Confidence Misses