# Can Large Language Models Design Biological Weapons? Evaluating Moremi Bio

arXiv: https://arxiv.org/abs/2505.17154 · HTML: https://mlsanity.com/paper/2505.17154/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 31 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 5 of 5)

- [ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learning](https://mlsanity.com/paper/2402.04268/) (2024)
- [Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools](https://mlsanity.com/paper/2306.13952/) (2023)
- [Can large language models democratize access to dual-use biotechnology?](https://mlsanity.com/paper/2306.03809/) (2023)
- [Advancing biomolecular understanding and design following human instructions](https://mlsanity.com/paper/2410.07919/) (2024)
- [Dual Use Concerns of Generative AI and Large Language Models](https://mlsanity.com/paper/2305.07882/) (2023)

## Built on by (most cited first, 1 of 1)

- [A Blind Spot in Alignment: Quantifying Biosecurity Risks in Large Language Models](https://mlsanity.com/paper/2608.02684/) (2026)

## Abstract (arXiv)

Advances in AI, particularly LLMs, have dramatically shortened drug discovery cycles by up to 40% and improved molecular target identification. However, these innovations also raise dual-use concerns by enabling the design of toxic compounds. Prompting Moremi Bio Agent without the safety guardrails to specifically design novel toxic substances, our study generated 1020 novel toxic proteins and 5,000 toxic small molecules. In-depth computational toxicity assessments revealed that all the proteins scored high in toxicity, with several closely matching known toxins such as ricin, diphtheria toxin, and disintegrin-based snake venom proteins. Some of these novel agents showed similarities with other several known toxic agents including disintegrin eristostatin, metalloproteinase, disintegrin triflavin, snake venom metalloproteinase, corynebacterium ulcerans toxin. Through quantitative risk assessments and scenario analyses, we identify dual-use capabilities in current LLM-enabled biodesign pipelines and propose multi-layered mitigation strategies. The findings from this toxicity assessment challenge claims that large language models (LLMs) are incapable of designing bioweapons. This reinforces concerns about the potential misuse of LLMs in biodesign, posing a significant threat to research and development (R&D). The accessibility of such technology to individuals with limited technical expertise raises serious biosecurity risks. Our findings underscore the critical need for robust governance and technical safeguards to balance rapid biotechnological innovation with biosecurity imperatives.