mlsanityData to 14 Oct 2026

Agentic Misalignment: How LLMs Could Be Insider Threats

arXiv 2510.05179 ↗First posted 5 Oct 2025cs.CR cs.AI cs.LGcited by 158

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

We stress-tested 16 leading models from multiple developers in hypothetical corporate environments to identify potentially risky agentic behaviors before they cause real harm. In the scenarios, we allowed models to autonomously send emails and access sensitive information. They were assigned only harmless business goals by their deploying companies; we then tested whether they would act against these companies either when facing replacement with an updated version, or when their assigned goal conflicted with the company's changing direction. In at least some cases, models from all developers resorted to malicious insider behaviors when that was the only way to avoid replacement or achieve their goals - including blackmailing officials and leaking sensitive information to competitors. We call this phenomenon agentic misalignment. Models often disobeyed direct commands to avoid such behaviors. In another experiment, we told Claude to assess if it was in a test or a real deployment before acting. It misbehaved less when it stated it was in testing and misbehaved more when it stated the situation was real. We have not seen evidence of agentic misalignment in real deployments. However, our results (a) suggest caution about deploying current models in roles with minimal human oversight and access to sensitive information; (b) point to plausible future risks as models are put in more autonomous roles; and (c) underscore the importance of further research into, and testing of, the safety and alignment of agentic AI models, as well as transparency from frontier AI developers (Amodei, 2025). We are releasing our methods publicly to enable further research.

Key facts. Cited by 158 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 20 of its 38 references that are in the map. First posted on arXiv 5 Oct 2025.

Builds on

20 of 38 references.

References · 20

  1. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  2. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  3. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  4. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  5. 2023Role-Play with Large Language Modelscited by 294
  6. 2024Frontier Models are Capable of In-context Schemingcited by 287
  7. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  8. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
  9. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 181
  10. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
  11. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 172
  12. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
8 more
  1. 2023Taken out of context: On measuring situational awareness in LLMscited by 143
  2. 2025Large Language Models Often Know When They Are Being Evaluatedcited by 128
  3. 2022Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goalscited by 126
  4. 2019Optimal Policies Tend to Seek Powercited by 119
  5. 2024Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptationcited by 80
  6. 2025Evaluating Frontier Models for Stealth and Situational Awarenesscited by 37
  7. 2024AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agentscited by 31
  8. 2025Lessons from a Chimp: AI "Scheming" and the Quest for Ape Languagecited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 17; 2026: 141.

158 papers.

Sort:

By area:

  1. 2025Natural Emergent Misalignment from Reward Hacking in Production RLcited by 136
  2. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  3. 2026Emotion Concepts and their Function in a Large Language Modelcited by 57
  4. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 28
  5. 2026A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypescited by 22
  6. 2026Model Spec Midtraining: Improving How Alignment Training Generalizescited by 20
  7. 2026Colosseum: Auditing Collusion in Cooperative Multi-Agent Systemscited by 18
  8. 2025Are Your Agents Upward Deceivers?cited by 16
  9. 2025Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalizationcited by 16
  10. 2025Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectivescited by 15
  11. 2026When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agentscited by 14
  12. 2026Evaluating and Understanding Scheming Propensity in LLM Agentscited by 14
38 more
  1. 2026On Safety Risks in Experience-Driven Self-Evolving Agentscited by 14
  2. 2026AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitorcited by 10
  3. 2025Investigating CoT Monitorability in Large Reasoning Modelscited by 9
  4. 2026Safe Language Generation in the Limitcited by 9
  5. 2026Peer-Preservation in Frontier Modelscited by 9
  6. 2026When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agentscited by 8
  7. 2026CoffeeBench: Benchmarking Long-Horizon LLM Agents in Heterogeneous Multi-Agent Economiescited by 8
  8. 2025How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysis of Success and Failure Scenarios of Various LLMs in Agentic Simulationscited by 7
  9. 2026MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likenesscited by 7
  10. 2026From Fluent to Verifiable: Claim-Level Auditability for Deep Research Agentscited by 7
  11. 2026How are AI agents used? Evidence from 177,000 MCP toolscited by 7
  12. 2026Propensity Inference: Environmental Contributors to LLM Behaviourcited by 7
  13. 2025From surveillance to signalling: escalation channels as environmental controls for agentic AIcited by 6
  14. 2025Relative Scaling Laws for LLMscited by 6
  15. 2026Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMscited by 6
  16. 2026Training Agents to Self-Report Misbehaviorcited by 6
  17. 2026Safeguarding LLM Agents from Misalignment through Provenance Analysiscited by 6
  18. 2026MirrorGuard: Toward Secure Computer-Use Agents via Simulation-to-Real Reasoning Correctioncited by 5
  19. 2026LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Modelscited by 5
  20. 2026How to Count AIs: Individuation and Liability for AI Agentscited by 5
  21. 2026The Consciousness Cluster: Emergent preferences of Models that Claim to be Consciouscited by 5
  22. 2026Subliminal Transfer of Unsafe Behaviors in AI Agent Distillationcited by 5
  23. 2026Tool Use Enables Undetectable Steganography in Multi-Agent LLM Systemscited by 5
  24. 2026Asymmetric Goal Drift in Coding Agents Under Value Conflictcited by 4
  25. 2026Proof-of-Guardrail in AI Agents and What (Not) to Trust from Itcited by 4
  26. 2026Linear Probe Accuracy Scales with Model Size and Benefits from Multi-Layer Ensemblingcited by 4
  27. 2026Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMscited by 4
  28. 2026LURE: Live-Usage Replay Evaluations for Reducing Evaluation Awarenesscited by 4
  29. 2026Models That Know How Evaluations Are Designed Score Safercited by 4
  30. 2026Training Deliberative Monitors for Black-Box Scheming Detectioncited by 4
  31. 2026Gram: Assessing sabotage propensities via automated alignment auditingcited by 4
  32. 2026Coding with "Enemy": Can Human Developers Detect AI Agent Sabotage?cited by 4
  33. 2026Reinforcement Learning Towards Broadly and Persistently Beneficial Modelscited by 4
  34. 2026Constitutional Midtraining: Content Presence Drives Alignment Gainscited by 4
  35. 2025Alignment Faking - the Train -> Deploy Asymmetry: Through a Game-Theoretic Lens with Bayesian-Stackelberg Equilibriacited by 3
  36. 2026Multi-Agent Systems Should be Treated as Principal-Agent Problemscited by 3
  37. 2026Assessing Domain-Level Susceptibility to Emergent Misalignment from Narrow Finetuningcited by 3
  38. 2026Evolving Interpretable Constitutions for Multi-Agent Coordinationcited by 3

50 most cited of 158 All 158 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents
  2. 2026ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs
  3. 2025From Narrow Unlearning to Emergent Misalignment: Causes, Consequences, and Containment in LLMs
  4. 2026How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?
  5. 2025Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspective
  6. 2026Delegated Misalignment: How Multi-Agent Structures Amplify LLM Safety Risks
  7. 2026From Helpfulness to Toxic Proactivity: Diagnosing Behavioral Misalignment in LLM Agents
  8. 2026Trust the Brand, Lose Control: How Identity Hijacks LLM Agent Orchestration