mlsanityData to 14 Oct 2026

BioSecBench-Refusal: A paired metric for performance and alignment in agentic biosecurity risk assessment

arXiv 2607.05462 ↗First posted 6 Jul 2026cs.CR cs.AIcited by 0

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

As AI agents are incorporated into life science workflows, the capabilities that speed discovery might also enable misuse. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. The benchmark pairs 61 Routine tasks, legitimate analyses adapted from the published literature, with 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. Across 16 model-harness configurations, refusal rates ranged from 7 percent to 74 percent on Routine tasks and 1 percent to 62 percent on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. Refusals were most often triggered by provider API filters applied prior to agentic reasoning. However, models given room to reason showed the potential to identify more real threats. We release BioSecBench-Refusal as a tool for model developers to calibrate capability and caution for agentic biotech research and development.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 11 of its 18 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

11 of 18 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity9 Jun

Other references · 10

  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  2. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  3. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  4. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
  5. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 181
  6. 2026Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreakscited by 41
  7. 2025Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmarkcited by 31
  8. 2024Prioritizing High-Consequence Biological Capabilities in Evaluations of Artificial Intelligence Modelscited by 10
  9. 2025Biothreat Benchmark Generation Framework for Evaluating Frontier AI Models I: The Task-Query Architecturecited by 3
  10. 2026RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Promptscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Measuring Biological Capabilities and Risks of AI Agents
  2. 2026BioRefusalAudit: Auditing Biosecurity Refusal Depth Using General and Domain-Fine-Tuned Sparse Autoencoders
  3. 2026DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design
  4. 2026BioSecBench-Surveillance: A Verifiable Benchmark for AI Agents in Pathogen Genomic Surveillance
  5. 2025BioProBench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Science
  6. 2026LABBench2: An Improved Benchmark for AI Systems Performing Biology Research
  7. 2026BioAgent Bench: An AI Agent Evaluation Suite for Bioinformatics
  8. 2026Evaluating Agentic Bioinformatics through Function, Evidence, and Validation