# BioSecBench-Refusal: A paired metric for performance and alignment in agentic biosecurity risk assessment

arXiv: https://arxiv.org/abs/2607.05462 · HTML: https://mlsanity.com/paper/2607.05462/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 11 of 11)

- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Safe RLHF: Safe Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2310.12773/) (2023)
- [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://mlsanity.com/paper/2403.03218/) (2024)
- [OR-Bench: An Over-Refusal Benchmark for Large Language Models](https://mlsanity.com/paper/2405.20947/) (2024)
- [Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming](https://mlsanity.com/paper/2501.18837/) (2025)
- [Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks](https://mlsanity.com/paper/2601.04603/) (2026)
- [Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark](https://mlsanity.com/paper/2504.16137/) (2025)
- [Prioritizing High-Consequence Biological Capabilities in Evaluations of Artificial Intelligence Models](https://mlsanity.com/paper/2407.13059/) (2024)
- [ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity](https://mlsanity.com/paper/2606.11150/) (2026)
- [Biothreat Benchmark Generation Framework for Evaluating Frontier AI Models I: The Task-Query Architecture](https://mlsanity.com/paper/2512.08130/) (2025)
- [RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts](https://mlsanity.com/paper/2605.21545/) (2026)

## Abstract (arXiv)

As AI agents are incorporated into life science workflows, the capabilities that speed discovery might also enable misuse. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. The benchmark pairs 61 Routine tasks, legitimate analyses adapted from the published literature, with 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. Across 16 model-harness configurations, refusal rates ranged from 7 percent to 74 percent on Routine tasks and 1 percent to 62 percent on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. Refusals were most often triggered by provider API filters applied prior to agentic reasoning. However, models given room to reason showed the potential to identify more real threats. We release BioSecBench-Refusal as a tool for model developers to calibrate capability and caution for agentic biotech research and development.