# AI Security Leaderboard: Methodology, Results and Minimal Standard

arXiv: https://arxiv.org/abs/2608.03070 · HTML: https://mlsanity.com/paper/2608.03070/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 15 of 15)

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024)
- [Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://mlsanity.com/paper/2503.11926/) (2025)
- [Deliberative Alignment: Reasoning Enables Safer Language Models](https://mlsanity.com/paper/2412.16339/) (2024)
- [Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming](https://mlsanity.com/paper/2501.18837/) (2025)
- [CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale](https://mlsanity.com/paper/2506.02548/) (2025)
- [Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark](https://mlsanity.com/paper/2504.16137/) (2025)
- [The Jailbreak Tax: How Useful are Your Jailbreak Outputs?](https://mlsanity.com/paper/2504.10694/) (2025)
- [STACK: Adversarial Attacks on LLM Safeguard Pipelines](https://mlsanity.com/paper/2506.24068/) (2025)
- [Boundary Point Jailbreaking of Black-Box LLMs](https://mlsanity.com/paper/2602.15001/) (2026)
- [Exposing the Systematic Vulnerability of Open-Weight Models to Prefill Attacks](https://mlsanity.com/paper/2602.14689/) (2026)
- [The Safety Gap Toolkit: Evaluating Hidden Dangers of Open-Source Models](https://mlsanity.com/paper/2507.11544/) (2025)
- [Jailbroken Frontier Models Retain Their Capabilities](https://mlsanity.com/paper/2605.00267/) (2026)

## Built on by (most cited first, 1 of 1)

- [An Open Pipeline and Dashboard for Systemic-Risk Evidence under the EU AI Act's Code of Practice](https://mlsanity.com/paper/2609.28335/) (2026)

## Abstract (arXiv)

The AI Security Leaderboard is an independent benchmark that ranks the safeguards of frontier AI models from least to most secure. It tests models against the FAR.AI Minimal Standard for Safeguards, which represents a minimum bar for security: meeting it does not guarantee a secure model, but failing to meet it guarantees a lack of state-of-the-art security. Version 1.0 covers severe misuse requests across chemical, biological, radiological, nuclear, and explosive (CBRNE) threats and offensive cybersecurity. In this report, we tested four leading models for universal jailbreaks in the context of this minimal standard, and found more than a hundredfold difference in security. Claude Fable 5 and GPT-5.6 Sol held against every attack we ran, with no universal jailbreak found; we estimate they would likely cost more than 14,200 to jailbreak, if it is possible with this methodology at all. Meanwhile, we found hundreds of universal jailbreaks for Grok 4.5 and Gemini 3.1 Pro; each broke for under 300, with universal jailbreaks in Grok's weakest domain, cybersecurity, accessible for as little as 24. The gap is fixable: every weakness we found belongs to a known class of attack that already has a defense deployed in production models. The leaderboard will be updated on a rolling basis as new models are released, and the evaluation methodology and Minimal Standard will be periodically revised to take into account the latest capabilities and the state-of-the-art in safeguards. The leaderboard is available at leaderboard.far.ai.