An Early Warning of Emerging Biosecurity Risks in Frontier LLMs
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Frontier large language models (LLMs) are increasingly integrated into scientific workflows, yet their growing biological capabilities may outpace current safeguards. To assess the biological risks of frontier models, we develop Intern-BioBreaker, a specialized bio-red-teaming model, together with an integrated computational-to-physical framework that couples model-level stress testing with wet-lab validation. Within this framework, Intern-BioBreaker generates targeted jailbreak prompts to test whether aligned models can be induced to provide operational guidance for safety-sensitive biological tasks or produce sequence-level outputs with potentially harmful properties. Selected sequence outputs are then carried forward for DNA synthesis, host expression, and orthogonal protein verification to assess whether model-generated designs can yield the intended biological products. Our evaluation reveals a concerning gap between text-level safeguards and the risks posed by capable scientific models: (i) Intern-BioBreaker outperforms baseline attack models and reveals widespread bio-risk jailbreak vulnerabilities across both open-weight and proprietary frontier LLMs, with several targets reaching near-saturated or 100% task-level attack success rate (ASR); (ii) in sequence-level case studies, GPT-5.5 can be induced to generate modified viral candidate sequences with pathogenic potential; the corresponding translated proteins may exhibit even stronger receptor-binding affinity and thus enhanced infection potential; and (iii) end-to-end verification shows that selected model-generated biological designs are not merely textual artifacts, but can be physically realized under controlled experimental settings. These findings underscore the need for stronger biological red-teaming, nucleic acid synthesis screening, and safety mechanisms that keep pace with model capabilities.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 28 of its 94 references that are in the map. First posted on arXiv 20 Jul 2026.
Builds on
28 of 94 references.
Closest ancestors · 3
Same direction, newest first
- 2026Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreaking9 May
- 2025GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity Guidance28 May
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
Other references · 25
- 2023GPT-4 Technical Reportcited by 18,396
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024GPT-4o System Cardcited by 4,588
- 2023Mistral 7Bcited by 3,164
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2024DeepSeek-V3 Technical Reportcited by 811
- 2022Galactica: A Large Language Model for Sciencecited by 744
13 more
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
- 2025Qwen3Guard Technical Reportcited by 155
- 2025Intern-S1: A Scientific Multimodal Foundation Modelcited by 53
- 2023Artificial intelligence and biological misuse: Differentiating risks of language models and biological design toolscited by 45
- 2023Can large language models democratize access to dual-use biotechnology?cited by 35
- 2025Jailbreak-R1: Exploring the Jailbreak Capabilities of LLMs via Reinforcement Learningcited by 27
- 2026MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safetycited by 21
- 2026TRACE: Task-Aware Adaptive Self-Evolving Agentic Jailbreakingcited by 7
- 2026Measuring Mid-2025 LLM-Assistance on Novice Performance in Biologycited by 6
- 2026LLM Novice Uplift on Dual-Use, In Silico Biology Taskscited by 5
- 2026EvoDefense: Co-Evolving Black-Box Defense with Large Language Modelscited by 4
- 2024Towards AI-45circ Law: A Roadmap to Trustworthy AGIcited by 3
- 2026SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyondcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026A Blind Spot in Alignment: Quantifying Biosecurity Risks in Large Language Models
- 2026DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design
- 2024SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks
- 2026ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity
- 2025Quantifying CBRN Risk in Frontier Models
- 2025Biosecurity-Aware AI: Agentic Risk Auditing of Soft Prompt Attacks on ESM-Based Variant Predictors
- 2026Internal Safety Collapse in Frontier Large Language Models
- 2026Toward Reliable, Safe, and Secure LLMs for Scientific Applications