mlsanityData to 14 Oct 2026

Detecting Strategic Deception Using Linear Probes

arXiv 2502.03407 ↗First posted 5 Feb 2025cs.LGcited by 68

On the map: Mechanistic interpretability → Probing deception, bias and beliefs

Abstract (arXiv)

AI models might use deceptive strategies as part of scheming or misaligned behaviour. Monitoring outputs alone is insufficient, since the AI might produce seemingly benign outputs while their internal reasoning is misaligned. We thus evaluate if linear probes can robustly detect deception by monitoring model activations. We test two probe-training datasets, one with contrasting instructions to be honest or deceptive (following Zou et al., 2023) and one of responses to simple roleplaying scenarios. We test whether these probes generalize to realistic settings where Llama-3.3-70B-Instruct behaves deceptively, such as concealing insider trading (Scheurer et al., 2023) and purposely underperforming on safety evaluations (Benton et al., 2024). We find that our probe distinguishes honest and deceptive responses with AUROCs between 0.96 and 0.999 on our evaluation datasets. If we set the decision threshold to have a 1% false positive rate on chat data not related to deception, our probe catches 95-99% of the deceptive responses. Overall we think white-box probes are promising for future monitoring systems, but current performance is insufficient as a robust defence against deception. Our probes' outputs can be viewed at data.apolloresearch.ai/dd and our code at github.com/ApolloResearch/deception-detection.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability. First posted on arXiv 5 Feb 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 33; 2026: 35.

68 papers.

Sort:

By area:

  1. 2025Detecting High-Stakes Interactions with Activation Probescited by 53
  2. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 38
  3. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 35
  4. 2025Among Us: A Sandbox for Measuring and Detecting Agentic Deceptioncited by 26
  5. 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoningcited by 26
  6. 2025RL-Obfuscation: Can Language Models Learn to Evade Latent-Space Monitors?cited by 24
  7. 2025Training LLMs for Honesty via Confessionscited by 22
  8. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 21
  9. 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 21
  10. 2025Benchmarking Deception Probes via Black-to-White Performance Boostscited by 19
  11. 2025Beyond Linear Probes: Dynamic Safety Monitoring for Language Modelscited by 16
  12. 2025Investigating task-specific prompts and sparse autoencoders for activation monitoringcited by 15
38 more
  1. 2025Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectivescited by 15
  2. 2025Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspectivecited by 13
  3. 2025Difficulties with Evaluating a Deception Detector for AIscited by 13
  4. 2026The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probescited by 12
  5. 2025Combining Cost-Constrained Runtime Monitors for AI Safetycited by 10
  6. 2025Preference Learning with Lie Detectors can Induce Honesty or Evasioncited by 9
  7. 2025Probe-Rewrite-Evaluate: A Workflow for Reliable Benchmarks and Quantifying Evaluation Awarenesscited by 9
  8. 2025The Impact of Off-Policy Training Data on Probe Generalisationcited by 9
  9. 2025Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitorscited by 9
  10. 2025PredictaBoard: Benchmarking LLM Score Predictabilitycited by 7
  11. 2025Can LLMs Lie? Investigation beyond Hallucinationcited by 7
  12. 2026Detecting Multi-Agent Collusion Through Multi-Agent Interpretabilitycited by 7
  13. 2025Caught in the Act: a mechanistic approach to detecting deceptioncited by 6
  14. 2025From surveillance to signalling: escalation channels as environmental controls for agentic AIcited by 6
  15. 2026Training Agents to Self-Report Misbehaviorcited by 6
  16. 2026Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probingcited by 6
  17. 2026Ensemble Monitoring for AI Control: Diverse Signals Outweigh More Computecited by 6
  18. 2026Mirage Probes: How Vision Models Fake Visual Understandingcited by 5
  19. 2025How Do LLMs Persuade? Linear Probes Can Uncover Persuasion Dynamics in Multi-Turn Conversationscited by 4
  20. 2026Linear Probe Accuracy Scales with Model Size and Benefits from Multi-Layer Ensemblingcited by 4
  21. 2026"Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organismscited by 4
  22. 2025Deceptive Automated Interpretability: Language Models Coordinating to Fool Oversight Systemscited by 3
  23. 2025Localized Calibrated Uncertainty in Code Language Modelscited by 3
  24. 2026Deception and Communication in Autonomous Multi-Agent Systems: An Experimental Study with Among Uscited by 3
  25. 2026An Independent Safety Evaluation of Kimi K2.5cited by 3
  26. 2026Probing Persona-Dependent Preferences in Language Modelscited by 3
  27. 2025Linear probes rely on textual evidence: Results from leakage mitigation studies in language modelscited by 2
  28. 2025Painless Activation Steering: An Automated, Lightweight Approach for Post-Training Large Language Modelscited by 2
  29. 2025Probe-based Fine-tuning for Reducing Toxicitycited by 2
  30. 2026Do LLMs Know What Is Private Internally? Probing and Steering Contextual Privacy Norms in Large Language Model Representationscited by 2
  31. 2026Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demandscited by 2
  32. 2026Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representationscited by 2
  33. 2026Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditingcited by 2
  34. 2026Emergent Collusion in Long-Horizon LLM Agent Interactioncited by 2
  35. 2025Wide Neural Networks as a Baseline for the Computational No-Coincidence Conjecturecited by 1
  36. 2025Red-teaming Activation Probes using Prompted LLMscited by 1
  37. 2026Why Safety Probes Catch Liars But Miss Fanaticscited by 1
  38. 2026Preference Instability in Reward Models: Detection and Mitigation via Sparse Autoencoderscited by 1

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026One Probe Won't Catch Them All: Towards Targeted Deception Detection
  2. 2026Caught in the Act: Probes Effectively Detect Sabotage and Catch Unverbalized Deception
  3. 2025The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blind
  4. 2026DeceptGuard :A Constitutional Oversight Framework For Detecting Deception in LLM Agents
  5. 2026DECOR: Auditing LLM Deception via Information Manipulation Theory
  6. 2026Lying to Win: Assessing LLM Deception through Human-AI Games and Parallel-World Probing
  7. 2026Decodability is Not Causality: Dissociating Probe Readouts from Behavioral Drivers via SAE Decomposition
  8. 2026DecepEval: A Benchmark for Evaluating Deception in LLM Agents