# Sabotage Evaluations for Frontier Models

arXiv: https://arxiv.org/abs/2410.21514 · HTML: https://mlsanity.com/paper/2410.21514/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 28 Oct 2024.

## Builds on (most cited first, 20 of 20)

- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://mlsanity.com/paper/2401.05566/) (2024)
- [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://mlsanity.com/paper/2403.03218/) (2024)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [Risks from Learned Optimization in Advanced Machine Learning Systems](https://mlsanity.com/paper/1906.01820/) (2019)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [An Overview of Catastrophic AI Risks](https://mlsanity.com/paper/2306.12001/) (2023)
- [Model evaluation for extreme risks](https://mlsanity.com/paper/2305.15324/) (2023)
- [Large Language Models can Strategically Deceive their Users when Put Under Pressure](https://mlsanity.com/paper/2311.07590/) (2023)
- [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024)
- [Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models](https://mlsanity.com/paper/2312.04724/) (2023)
- [Evaluating Frontier Models for Dangerous Capabilities](https://mlsanity.com/paper/2403.13793/) (2024)
- [Scheming AIs: Will AIs fake alignment during training in order to get power?](https://mlsanity.com/paper/2311.08379/) (2023)
- [Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs](https://mlsanity.com/paper/2407.04694/) (2024)
- [Safety Cases: How to Justify the Safety of Advanced AI Systems](https://mlsanity.com/paper/2403.10462/) (2024)
- [Stress-Testing Capability Elicitation With Password-Locked Models](https://mlsanity.com/paper/2405.19550/) (2024)
- [Measuring and Benchmarking Large Language Models' Capabilities to Generate Persuasive Language](https://mlsanity.com/paper/2406.17753/) (2024)
- [TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI](https://mlsanity.com/paper/2306.06924/) (2023)
- [Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance](https://mlsanity.com/paper/2407.07950/) (2024)

## Built on by (most cited first, 30 of 52)

- [Alignment faking in large language models](https://mlsanity.com/paper/2412.14093/) (2024)
- [Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety](https://mlsanity.com/paper/2507.11473/) (2025)
- [RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts](https://mlsanity.com/paper/2411.15114/) (2024)
- [Natural Emergent Misalignment from Reward Hacking in Production RL](https://mlsanity.com/paper/2511.18397/) (2025)
- [When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors](https://mlsanity.com/paper/2507.05246/) (2025)
- [A sketch of an AI control safety case](https://mlsanity.com/paper/2501.17315/) (2025)
- [Evaluating Frontier Models for Stealth and Situational Awareness](https://mlsanity.com/paper/2505.01420/) (2025)
- [Probing and Steering Evaluation Awareness of Language Models](https://mlsanity.com/paper/2507.01786/) (2025)
- [Reliable Weak-to-Strong Monitoring of LLM Agents](https://mlsanity.com/paper/2508.19461/) (2025)
- [The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awareness](https://mlsanity.com/paper/2505.14617/) (2025)
- [Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models](https://mlsanity.com/paper/2412.01784/) (2024)
- [The Elicitation Game: Evaluating Capability Elicitation Techniques](https://mlsanity.com/paper/2502.02180/) (2025)
- [LLMs Can Covertly Sandbag on Capability Evaluations Against Chain-of-Thought Monitoring](https://mlsanity.com/paper/2508.00943/) (2025)
- [Evaluation Faking: Unveiling Observer Effects in Safety Evaluation of Frontier AI Systems](https://mlsanity.com/paper/2505.17815/) (2025)
- [Monitoring Emergent Reward Hacking During Generation via Internal Activations](https://mlsanity.com/paper/2603.04069/) (2026)
- [The Singapore Consensus on Global AI Safety Research Priorities](https://mlsanity.com/paper/2506.20702/) (2025)
- [Combining Cost-Constrained Runtime Monitors for AI Safety](https://mlsanity.com/paper/2507.15886/) (2025)
- [Do Large Language Models Know What They Are Capable Of?](https://mlsanity.com/paper/2512.24661/) (2025)
- [The Impact of Off-Policy Training Data on Probe Generalisation](https://mlsanity.com/paper/2511.17408/) (2025)
- [Measuring AI R&D Automation](https://mlsanity.com/paper/2603.03992/) (2026)
- [The Truthfulness Spectrum Hypothesis](https://mlsanity.com/paper/2602.20273/) (2026)
- [LinuxArena: A Control Setting for AI Agents in Live Production Software Environments](https://mlsanity.com/paper/2604.15384/) (2026)
- [Evaluation Awareness in Language Models Has Limited Effect on Behaviour](https://mlsanity.com/paper/2605.05835/) (2026)
- [Advanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level III](https://mlsanity.com/paper/2507.02954/) (2025)
- [Training Agents to Self-Report Misbehavior](https://mlsanity.com/paper/2602.22303/) (2026)
- [OpenDeception: Learning Deception and Trust in Human-AI Interaction via Multi-Agent Simulation](https://mlsanity.com/paper/2504.13707/) (2025)
- [Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Framework](https://mlsanity.com/paper/2507.12872/) (2025)
- [Distributed Attacks in Persistent-State AI Control](https://mlsanity.com/paper/2607.02514/) (2026)
- [Removing Sandbagging in LLMs by Training with Weak Supervision](https://mlsanity.com/paper/2604.22082/) (2026)
- [Training Deliberative Monitors for Black-Box Scheming Detection](https://mlsanity.com/paper/2605.29601/) (2026)

## Abstract (arXiv)

Sufficiently capable models could subvert human oversight and decision-making in important contexts. For example, in the context of AI development, models could covertly sabotage efforts to evaluate their own dangerous capabilities, to monitor their behavior, or to make decisions about their deployment. We refer to this family of abilities as sabotage capabilities. We develop a set of related threat models and evaluations. These evaluations are designed to provide evidence that a given model, operating under a given set of mitigations, could not successfully sabotage a frontier model developer or other large organization's activities in any of these ways. We demonstrate these evaluations on Anthropic's Claude 3 Opus and Claude 3.5 Sonnet models. Our results suggest that for these models, minimal mitigations are currently sufficient to address sabotage risks, but that more realistic evaluations and stronger mitigations seem likely to be necessary soon as capabilities improve. We also survey related evaluations we tried and abandoned. Finally, we discuss the advantages of mitigation-aware capability evaluations, and of simulating large-scale deployments using small-scale statistics.