# ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity

arXiv: https://arxiv.org/abs/2606.11150 · HTML: https://mlsanity.com/paper/2606.11150/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 9 Jun 2026.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 16 of 16)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Rethinking Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2402.08787/) (2024)
- [Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models](https://mlsanity.com/paper/2408.08926/) (2024)
- [LAB-Bench: Measuring Capabilities of Language Models for Biology Research](https://mlsanity.com/paper/2407.10362/) (2024)
- [CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark](https://mlsanity.com/paper/2409.11363/) (2024)
- [DiscoveryBench: Towards Data-Driven Discovery with Large Language Models](https://mlsanity.com/paper/2407.01725/) (2024)
- [BixBench: a Comprehensive Benchmark for LLM-based Agents in Computational Biology](https://mlsanity.com/paper/2503.00096/) (2025)
- [CRISPR-GPT for Agentic Automation of Gene-editing Experiments](https://mlsanity.com/paper/2404.18021/) (2024)
- [BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments](https://mlsanity.com/paper/2405.17631/) (2024)
- [Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark](https://mlsanity.com/paper/2504.16137/) (2025)
- [BioCoder: A Benchmark for Bioinformatics Code Generation with Large Language Models](https://mlsanity.com/paper/2308.16458/) (2023)
- [Align-Pro: A Principled Approach to Prompt Optimization for LLM Alignment](https://mlsanity.com/paper/2501.03486/) (2025)
- [GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity Guidance](https://mlsanity.com/paper/2505.23839/) (2025)
- [UniSite: The First Cross-Structure Dataset and Learning Framework for End-to-End Ligand Binding Site Detection](https://mlsanity.com/paper/2506.03237/) (2025)

## Built on by (most cited first, 6 of 6)

- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [A Blind Spot in Alignment: Quantifying Biosecurity Risks in Large Language Models](https://mlsanity.com/paper/2608.02684/) (2026)
- [Safeguard-Conditioned Uplift: Measuring Utility-Risk Frontiers for Dual-Use Biology Assistants](https://mlsanity.com/paper/2607.13039/) (2026)
- [BioSecBench-Refusal: A paired metric for performance and alignment in agentic biosecurity risk assessment](https://mlsanity.com/paper/2607.05462/) (2026)
- [Artificial intelligence and biosecurity: capabilities, threat pathways, and defense-in-depth governance](https://mlsanity.com/paper/2609.16213/) (2026)
- [DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design](https://mlsanity.com/paper/2610.13116/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are rapidly acquiring capabilities relevant to biological research, from literature synthesis to interpretation of experimental data. Increasingly, LLM agents can also perform in silico biology tasks that previously required experienced human biologists. These emerging AI capabilities offer new opportunities for scientific discovery and biomedical advances, but they also shift the landscape of biosecurity risks. To address this, we introduce the Agentic Bio-Capabilities Benchmark (ABC-Bench), a suite of tasks to measure agentic biosecurity-relevant capabilities. ABC-Bench evaluates LLM agents on both benign and dual-use biology tasks: writing code to operate liquid handling robots, designing DNA fragments for in vitro assembly, and evading DNA synthesis screening. These tasks require a combination of biology and software expertise. All tested LLM agents outperformed the median expert human baseliner on all three tasks. Agents performed highly on tasks drawing on published knowledge and well-documented protocols, and more weakly on a task requiring novel bioinformatics reasoning. In three wet-lab validation experiments, we found that OpenAI's o4-mini-high produced scripts that, when run on an OpenTrons liquid handling robot, successfully assembled DNA with expected sequences.