# Evaluating Frontier Models for Dangerous Capabilities

arXiv: https://arxiv.org/abs/2403.13793 · HTML: https://mlsanity.com/paper/2403.13793/

## Key facts

- Cited by 128 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2024.

## Built on by (most cited first, 30 of 128)

- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [Measuring AI Ability to Complete Long Software Tasks](https://mlsanity.com/paper/2503.14499/) (2025)
- [AI Sandbagging: Language Models can Strategically Underperform on Evaluations](https://mlsanity.com/paper/2406.07358/) (2024)
- [LLM Agents can Autonomously Exploit One-day Vulnerabilities](https://mlsanity.com/paper/2404.08144/) (2024)
- [Measuring what Matters: Construct Validity in Large Language Model Benchmarks](https://mlsanity.com/paper/2511.04703/) (2025)
- [Best-of-N Jailbreaking](https://mlsanity.com/paper/2412.03556/) (2024)
- [SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents](https://mlsanity.com/paper/2506.15740/) (2025)
- [Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents](https://mlsanity.com/paper/2410.13886/) (2024)
- [Introducing v0.5 of the AI Safety Benchmark from MLCommons](https://mlsanity.com/paper/2404.12241/) (2024)
- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [EnIGMA: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities](https://mlsanity.com/paper/2409.16165/) (2024)
- [Sabotage Evaluations for Frontier Models](https://mlsanity.com/paper/2410.21514/) (2024)
- [LatentQA: Teaching LLMs to Decode Activations Into Natural Language](https://mlsanity.com/paper/2412.08686/) (2024)
- [Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation](https://mlsanity.com/paper/2410.03415/) (2024)
- [Stress-Testing Capability Elicitation With Password-Locked Models](https://mlsanity.com/paper/2405.19550/) (2024)
- [A sketch of an AI control safety case](https://mlsanity.com/paper/2501.17315/) (2025)
- [Evaluating Frontier Models for Stealth and Situational Awareness](https://mlsanity.com/paper/2505.01420/) (2025)
- [Towards evaluations-based safety cases for AI scheming](https://mlsanity.com/paper/2411.03336/) (2024)
- [Adversaries Can Misuse Combinations of Safe Models](https://mlsanity.com/paper/2406.14595/) (2024)
- [Reliable Weak-to-Strong Monitoring of LLM Agents](https://mlsanity.com/paper/2508.19461/) (2025)
- [Representation Bending for Large Language Model Safety](https://mlsanity.com/paper/2504.01550/) (2025)
- [The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems](https://mlsanity.com/paper/2602.17753/) (2026)
- [Towards interactive evaluations for interaction harms in human-AI systems](https://mlsanity.com/paper/2405.10632/) (2024)
- [ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?](https://mlsanity.com/paper/2605.11086/) (2026)
- [Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agents](https://mlsanity.com/paper/2502.11355/) (2025)
- [Lessons From Red Teaming 100 Generative AI Products](https://mlsanity.com/paper/2501.07238/) (2025)
- [Cyber-Zero: Training Cybersecurity Agents without Runtime](https://mlsanity.com/paper/2508.00910/) (2025)
- [PropensityBench: Evaluating Latent Safety Risks in Large Language Models via an Agentic Approach](https://mlsanity.com/paper/2511.20703/) (2025)

## Abstract (arXiv)

To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four areas: (1) persuasion and deception; (2) cyber-security; (3) self-proliferation; and (4) self-reasoning. We do not find evidence of strong dangerous capabilities in the models we evaluated, but we flag early warning signs. Our goal is to help advance a rigorous science of dangerous capability evaluation, in preparation for future models.