# Quantifying CBRN Risk in Frontier Models

arXiv: https://arxiv.org/abs/2510.21133 · HTML: https://mlsanity.com/paper/2510.21133/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 24 Oct 2025.

## Builds on (most cited first, 9 of 9)

- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [DeepInception: Hypnotize Large Language Model to Be Jailbreaker](https://mlsanity.com/paper/2311.03191/) (2023)
- [Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment](https://mlsanity.com/paper/2308.09662/) (2023)
- [SafetyBench: Evaluating the Safety of Large Language Models](https://mlsanity.com/paper/2309.07045/) (2023)
- [SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models](https://mlsanity.com/paper/2410.18927/) (2024)
- [WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models](https://mlsanity.com/paper/2408.03837/) (2024)
- [Safety by Measurement: A Systematic Literature Review of AI Safety Evaluation Methods](https://mlsanity.com/paper/2505.05541/) (2025)
- [SAGE-RT: Synthetic Alignment data Generation for Safety Evaluation and Red Teaming](https://mlsanity.com/paper/2408.11851/) (2024)
- [Measurement challenges in AI catastrophic risk governance and safety frameworks](https://mlsanity.com/paper/2410.00608/) (2024)

## Built on by (most cited first, 6 of 6)

- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Trojan-Speak: Bypassing Constitutional Classifiers with No Jailbreak Tax via Adversarial Finetuning](https://mlsanity.com/paper/2603.29038/) (2026)
- [SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring](https://mlsanity.com/paper/2607.18665/) (2026)
- [VET: A Framework for Analyzing AI Discourse](https://mlsanity.com/paper/2606.01929/) (2026)
- [A Threshold Exceedance Framework for CBRN Uplift Evaluation in Frontier Language Models](https://mlsanity.com/paper/2607.12200/) (2026)
- [AI Models Can Provably Hide Arbitrary Capabilities](https://mlsanity.com/paper/2610.13928/) (2026)

## Abstract (arXiv)

Frontier Large Language Models (LLMs) pose unprecedented dual-use risks through the potential proliferation of chemical, biological, radiological, and nuclear (CBRN) weapons knowledge. We present the first comprehensive evaluation of 10 leading commercial LLMs against both a novel 200-prompt CBRN dataset and a 180-prompt subset of the FORTRESS benchmark, using a rigorous three-tier attack methodology. Our findings expose critical safety vulnerabilities: Deep Inception attacks achieve 86.0% success versus 33.8% for direct requests, demonstrating superficial filtering mechanisms; Model safety performance varies dramatically from 2% (claude-opus-4) to 96% (mistral-small-latest) attack success rates; and eight models exceed 70% vulnerability when asked to enhance dangerous material properties. We identify fundamental brittleness in current safety alignment, where simple prompt engineering techniques bypass safeguards for dangerous CBRN information. These results challenge industry safety claims and highlight urgent needs for standardized evaluation frameworks, transparent safety metrics, and more robust alignment techniques to mitigate catastrophic misuse risks while preserving beneficial capabilities.