mlsanityData to 7 Oct 2026

Activation Probes Surface Code-Security Signals that the Model's Output Misses

arXiv 2608.09643 ↗First posted 10 Aug 2026cs.CR cs.LGcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

AI coding agents now write a growing share of production code, and human security review does not scale at the rate code is generated. The agents in widest use are closed-weight, so a deploying team cannot read their internals. It can instead run an open-weight model as a reviewer over the agent's output. That reviewer's activations are readable. We ask whether reading those activations recovers a security signal that simply asking the same reviewer misses. We fit a single linear probe per model on a corpus of paired vulnerable-and-fixed Python functions, then test it without retraining on real disclosed vulnerabilities whose weakness type the probe never saw in training, across five open-weight reviewer models. On the vulnerabilities fixed by changing a single function, the probe scores the vulnerable function above its fix on 61-67% of cases for every model, beating the 50% chance line. It also beats the same model's prompted YES/NO win-rate read from its logits, under every prompt we try. Asking the model for a written verdict, even with chain-of-thought, returns the same answer on the vulnerable and fixed function most of the time and so cannot tell them apart. Model activations carry a code-security signal that prompting the same model misses.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 20 of its 26 references that are in the map. First posted on arXiv 10 Aug 2026.

Builds on

20 of 26 references.

References · 20

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  5. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  6. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  7. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  8. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  9. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  10. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 149
  11. 2024Vulnerability Detection with Code Language Models: How Far Are We?cited by 133
  12. 2018The Adverse Effects of Code Duplication in Machine Learning Models of Codecited by 123
8 more
  1. 2023LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarkscited by 94
  2. 2025Detecting High-Stakes Interactions with Activation Probescited by 45
  3. 2024NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internalscited by 43
  4. 2023Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Studycited by 36
  5. 2025PATCHEVAL: A New Benchmark for Evaluating LLMs on Patching Real-World Vulnerabilitiescited by 19
  6. 2025CASTLE: Benchmarking Dataset for Static Code Analyzers and LLMs towards CWE Detectioncited by 10
  7. 2025LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMscited by 4
  8. 2025AI Code in the Wild: Measuring Security Risks and Ecosystem Shifts of AI-Generated Code in Modern Softwarecited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026You Can't Escape Your Own Activations : Evaluation Awareness and Multi-Agent Monitoring
  2. 2026From Probe Scores to Alarm Policies: Operational Validity of Activation Monitors for Language-Model Agents
  3. 2026Prompted to Discriminate: Generalizing Malicious-Input Probes in the Wild
  4. 2026Beyond Pass@k: Measuring Reliability and Security of Agentic Code Generation
  5. 2026Building to the Test: Coding Agents Deliver What You Check, Not What You Requested
  6. 2026Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure
  7. 2026Reward Hacking Challenges Oversight of Autonomous Research Agents
  8. 2026A Near-Zero Monitor Readout Is Not Evidence of Behavioral Control