mlsanityData to 7 Oct 2026

Evaluating the Potential of Leading Large Language Models in Reasoning Biology Questions

arXiv 2311.07582 ↗First posted 5 Nov 2023cs.CL cs.AIcited by 7

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Recent advances in Large Language Models (LLMs) have presented new opportunities for integrating Artificial General Intelligence (AGI) into biological research and education. This study evaluated the capabilities of leading LLMs, including GPT-4, GPT-3.5, PaLM2, Claude2, and SenseNova, in answering conceptual biology questions. The models were tested on a 108-question multiple-choice exam covering biology topics in molecular biology, biological techniques, metabolic engineering, and synthetic biology. Among the models, GPT-4 achieved the highest average score of 90 and demonstrated the greatest consistency across trials with different prompts. The results indicated GPT-4's proficiency in logical reasoning and its potential to aid biology research through capabilities like data analysis, hypothesis generation, and knowledge integration. However, further development and validation are still required before the promise of LLMs in accelerating biological discovery can be realized.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 1 of its 13 references that are in the map. First posted on arXiv 5 Nov 2023.

Builds on

1 of 13 references.

References · 1

  1. 2023Improving Code Generation by Training with Natural Language Feedbackcited by 75

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 2.

7 papers.

  1. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
  2. 2024Revolutionizing Finance with LLMs: An Overview of Applications and Insightscited by 88
  3. 2024Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysiscited by 21
  4. 2024Assessing Large Language Models in Mechanical Engineering Education: A Study on Mechanics-Focused Conceptual Understandingcited by 15
  5. 2025Large Language Models for Bioinformaticscited by 3
  6. 2025CellTypeAgent: Trustworthy cell type annotation with Large Language Modelscited by 1
  7. 2024Survey on Reasoning Capabilities and Accessibility of Large Language Models Using Biology-related Questionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Research
  2. 2025RoBiologyDataChoiceQA: A Romanian Dataset for improving Biology understanding of Large Language Models
  3. 2023GPT-4 as an Agronomist Assistant? Answering Agriculture Exams Using Large Language Models
  4. 2025Benchmarking Large Language Models on Multiple Tasks in Bioinformatics NLP with Prompting
  5. 2024SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models
  6. 2025Toward Scientific Reasoning in LLMs: Training from Expert Discussions via Reinforcement Learning
  7. 2026BioEVAL: A global, multi-institutional benchmark of large language and multimodal models for bioengineering
  8. 2023GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Information