Towards Responsible Governance of Biological Design Tools
Not on the map. Why?
Abstract (arXiv)
Recent advancements in generative machine learning have enabled rapid progress in biological design tools (BDTs) such as protein structure and sequence prediction models. The unprecedented predictive accuracy and novel design capabilities of BDTs present new and significant dual-use risks. For example, their predictive accuracy allows biological agents, whether vaccines or pathogens, to be developed more quickly, while the design capabilities could be used to discover drugs or evade DNA screening techniques. Similar to other dual-use AI systems, BDTs present a wicked problem: how can regulators uphold public safety without stifling innovation? We highlight how current regulatory proposals that are primarily tailored toward large language models may be less effective for BDTs, which require fewer computational resources to train and are often developed in an open-source manner. We propose a range of measures to mitigate the risk that BDTs are misused, across the areas of responsible development, risk assessment, transparency, access management, cybersecurity, and investing in resilience. Implementing such measures will require close coordination between developers and governments.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 17 of its 57 references that are in the map. First posted on arXiv 27 Nov 2023.
Builds on
17 of 57 references.
References · 17
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2018Model Cards for Model Reportingcited by 1,068
- 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
- 2022Predictability and Surprise in Large Generative Modelscited by 209
- 2023Model evaluation for extreme riskscited by 167
- 2019Thieves on Sesame Street! Model Extraction of BERT-based APIscited by 136
- 2023Frontier AI Regulation: Managing Emerging Risks to Public Safetycited by 95
- 2023The Gradient of Generative AI Release: Methods and Considerationscited by 63
- 2023Artificial intelligence and biological misuse: Differentiating risks of language models and biological design toolscited by 43
- 2022Structured access: an emerging paradigm for safe AI deploymentcited by 40
- 2023Can large language models democratize access to dual-use biotechnology?cited by 33
5 more
- 2023Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectivescited by 31
- 2023Towards best practices in AGI safety and governance: A survey of expert opinioncited by 27
- 2023Will releasing the weights of future large language models grant widespread access to pandemic agents?cited by 25
- 2023What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoringcited by 20
- 2023Censoring chemical data to mitigate dual use riskcited by 3
Built on by
Citing papers per year: 2024: 2; 2025: 1; 2026: 2.
5 papers.
- 2024Are large language models superhuman chemists?cited by 34
- 2024The Reality of AI and Bioriskcited by 8
- 2025From Turing to Tomorrow: The UK's Approach to AI Regulationcited by 0
- 2026Know Your Scientist: KYC as Biosecurity Infrastructurecited by 0
- 2026Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Toolscited by 0
- 2026Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools5 Sep 2026
- 2026Know Your Scientist: KYC as Biosecurity Infrastructure5 Feb 2026
- 2025From Turing to Tomorrow: The UK's Approach to AI Regulation3 Jul 2025
- 2024The Reality of AI and Biorisk2 Dec 2024
- 2024Are large language models superhuman chemists?1 Apr 2024