mlsanityData to 14 Oct 2026

Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations

arXiv 2605.27958 ↗First posted 27 May 2026cs.CL cs.AI cs.LGcited by 2

On the map: Mechanistic interpretability → Probing deception, bias and beliefs

Abstract (arXiv)

Linear probes trained on LLM activations are increasingly proposed as deception-detection metrics, yet report AUROC exceeding 0.96 on clean benchmarks while collapsing under distributional shift. This paper systematically pressure-tests probe-based metrics across the Gemma 3 model family (1B-27B parameters), diagnosing why they fail rather than merely documenting that they fail. We test four hypotheses about deception encoding: (1) single linear direction, (2) multi-dimensional subspace, (3) convex conic hull, (4) entropy proxy. Our design includes cross-domain transfer matrices, multi-dimensional probe analysis with permutation null baselines, entropy-residualization tests, and distractor evaluations across 8 stylistic shifts. We find that: (a) probes achieve near-perfect AUROC (>=0.998) on clean data but collapse under stylistic shifts; style-augmented probes recover near-perfect detection (mean AUROC 0.979-0.983) on unseen styles; (b) the single-direction hypothesis is rejected (k=1 captures only 0.61-0.80 AUROC), with cross-domain transfer failure confirmed as geometric rather than layer-mismatch-driven; (c) the entropy-proxy hypothesis is rejected (max |rho|=0.454, max Delta-AUROC after residualization=0.004); and (d) deception does not form a significant linear subspace (per-domain k*=0), yet multi-dimensional probes (k>=5) recover the signal through distributed sub-threshold features. Probe fragility reflects distributional narrowness rather than an architectural limitation: style-augmented probes recover near-perfect detection at both 4B and 27B, establishing that the inverse scaling pattern is a training-distribution artifact rather than a genuine scale-dependent phenomenon.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability. Builds on 14 of its 22 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

14 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Detecting Strategic Deception Using Linear Probes5 Feb

Other references · 13

  1. 2020GLU Variants Improve Transformercited by 1,804
  2. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  3. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  4. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  5. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  6. 2020Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metricscited by 158
  7. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 83
  8. 2024Truth is Universal: Robust Detection of Lies in LLMscited by 79
  9. 2024Obfuscated Activations Bypass LLM Latent-Space Defensescited by 45
  10. 2025RL-Obfuscation: Can Language Models Learn to Evade Latent-Space Monitors?cited by 24
  11. 2025Auditing Games for Sandbaggingcited by 14
  12. 2025Caught in the Act: a mechanistic approach to detecting deceptioncited by 6
1 more
  1. 2025From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Liars' Bench: The Impact of Lie Typology, Depth, and Sparsity on Deception Detection in LLMs
  2. 2025Hidden in Plain Sight: Evaluation of the Deception Detection Capabilities of LLMs in Multimodal Settings
  3. 2026One Probe Won't Catch Them All: Towards Targeted Deception Detection
  4. 2026Probe Generalization as Subspace Selection for OOD Deception Detection
  5. 2026Caught in the Act: Probes Effectively Detect Sabotage and Catch Unverbalized Deception
  6. 2026Linear Probe Accuracy Scales with Model Size and Benefits from Multi-Layer Ensembling
  7. 2026DECOR: Auditing LLM Deception via Information Manipulation Theory
  8. 2025Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts