# Measuring Mid-2025 LLM-Assistance on Novice Performance in Biology

arXiv: https://arxiv.org/abs/2602.16703 · HTML: https://mlsanity.com/paper/2602.16703/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 60 references that are in the map.
- First posted on arXiv 18 Feb 2026.

## Builds on (most cited first, 18 of 18)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://mlsanity.com/paper/2403.03218/) (2024)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [LAB-Bench: Measuring Capabilities of Language Models for Biology Research](https://mlsanity.com/paper/2407.10362/) (2024)
- [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://mlsanity.com/paper/1507.04544/) (2015)
- [Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity](https://mlsanity.com/paper/2507.09089/) (2025)
- [The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas](https://mlsanity.com/paper/2506.20803/) (2025)
- [CRISPR-GPT for Agentic Automation of Gene-editing Experiments](https://mlsanity.com/paper/2404.18021/) (2024)
- [Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools](https://mlsanity.com/paper/2306.13952/) (2023)
- [Can large language models democratize access to dual-use biotechnology?](https://mlsanity.com/paper/2306.03809/) (2023)
- [Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark](https://mlsanity.com/paper/2504.16137/) (2025)
- [Estimating Worst-Case Frontier Risks of Open-Weight LLMs](https://mlsanity.com/paper/2508.03153/) (2025)
- [LLMs Outperform Experts on Challenging Biology Benchmarks](https://mlsanity.com/paper/2505.06108/) (2025)
- [LabOS: The AI-XR Co-Scientist That Sees and Works With Humans](https://mlsanity.com/paper/2510.14861/) (2025)
- [Advancing Science- and Evidence-based AI Policy](https://mlsanity.com/paper/2508.02748/) (2025)
- [AI Knowledge and Reasoning: Emulating Expert Creativity in Scientific Research](https://mlsanity.com/paper/2404.04436/) (2024)
- [Measuring skill-based uplift from AI in a real biological laboratory](https://mlsanity.com/paper/2512.10960/) (2025)

## Built on by (most cited first, 6 of 6)

- [Life After Benchmark Saturation: A Case Study of CORE-Bench](https://mlsanity.com/paper/2606.26158/) (2026)
- [Safeguard-Conditioned Uplift: Measuring Utility-Risk Frontiers for Dual-Use Biology Assistants](https://mlsanity.com/paper/2607.13039/) (2026)
- [An Early Warning of Emerging Biosecurity Risks in Frontier LLMs](https://mlsanity.com/paper/2607.18056/) (2026)
- [Risk Reporting for Developers' Internal AI Model Use](https://mlsanity.com/paper/2604.24966/) (2026)
- [A Threshold Exceedance Framework for CBRN Uplift Evaluation in Frontier Language Models](https://mlsanity.com/paper/2607.12200/) (2026)
- [Artificial intelligence and biosecurity: capabilities, threat pathways, and defense-in-depth governance](https://mlsanity.com/paper/2609.16213/) (2026)

## Abstract (arXiv)

Large language models (LLMs) perform strongly on biological benchmarks, raising concerns that they may help novice actors acquire dual-use laboratory skills. Yet, whether this translates to improved human performance in the physical laboratory remains unclear. To address this, we conducted a pre-registered, investigator-blinded, randomized controlled trial (June-August 2025; n = 153) evaluating whether LLMs improve novice performance in tasks that collectively model a viral reverse genetics workflow. We observed no significant difference in the primary endpoint of workflow completion (5.2% LLM vs. 6.6% Internet; P = 0.759), nor in the success rate of individual tasks. However, the LLM arm had numerically higher success rates in four of the five tasks, most notably for the cell culture task (68.8% LLM vs. 55.3% Internet; P = 0.059). Post-hoc Bayesian modeling of pooled data estimates an approximate 1.4-fold increase (95% CrI 0.74-2.62) in success for a "typical" reverse genetics task under LLM assistance. Ordinal regression modelling suggests that participants in the LLM arm were more likely to progress through intermediate steps across all tasks (posterior probability of a positive effect: 81%-96%). Overall, mid-2025 LLMs did not substantially increase novice completion of complex laboratory procedures but were associated with a modest performance benefit. These results reveal a gap between in silico benchmarks and real-world utility, underscoring the need for physical-world validation of AI biosecurity assessments as model capabilities and user proficiency evolve.