# Evaluation of large language models for discovery of gene set function

arXiv: https://arxiv.org/abs/2309.04019 · HTML: https://mlsanity.com/paper/2309.04019/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 125 references that are in the map.
- First posted on arXiv 7 Sep 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [Boosting Theory-of-Mind Performance in Large Language Models via Prompting](https://mlsanity.com/paper/2304.11490/) (2023)
- [DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents](https://mlsanity.com/paper/2303.17071/) (2023)
- [Structured prompt interrogation and recursive extraction of semantics (SPIRES): A method for populating knowledge bases using zero-shot learning](https://mlsanity.com/paper/2304.02711/) (2023)
- [GPT-3.5, GPT-4, or BARD? Evaluating LLMs Reasoning Ability in Zero-Shot Setting and Performance Boosting Through Prompts](https://mlsanity.com/paper/2305.12477/) (2023)
- [Gene Set Summarization using Large Language Models](https://mlsanity.com/paper/2305.13338/) (2023)
- [An automatically discovered chain-of-thought prompt generalizes to novel models and datasets](https://mlsanity.com/paper/2305.02897/) (2023)

## Built on by (most cited first, 14 of 14)

- [Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving](https://mlsanity.com/paper/2405.12205/) (2024)
- [BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments](https://mlsanity.com/paper/2405.17631/) (2024)
- [Thought Graph: Generating Thought Process for Biological Reasoning](https://mlsanity.com/paper/2403.07144/) (2024)
- [GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain Databases](https://mlsanity.com/paper/2405.16205/) (2024)
- [scDrugMap: Benchmarking Large Foundation Models for Drug Response Prediction](https://mlsanity.com/paper/2505.05612/) (2025)
- [Gene-R1: Reasoning with Data-Augmented Lightweight LLMs for Gene Set Analysis](https://mlsanity.com/paper/2509.10575/) (2025)
- [GPTON: Generative Pre-trained Transformers enhanced with Ontology Narration for accurate annotation of biological data](https://mlsanity.com/paper/2410.10899/) (2024)
- [HypoGeneAgent: A Hypothesis Language Agent for Gene-Set Cluster Resolution Selection Using Perturb-seq Datasets](https://mlsanity.com/paper/2509.09740/) (2025)
- [zERExtractor:An Automated Platform for Enzyme-Catalyzed Reaction Data Extraction from Scientific Literature](https://mlsanity.com/paper/2508.09995/) (2025)
- [BRAINCELL-AID: An Agentic AI Created Brain Cell Type Resource for Community Annotation](https://mlsanity.com/paper/2510.17064/) (2025)
- [PRISM: PRIor from corpus Statistics for topic Modeling](https://mlsanity.com/paper/2603.29406/) (2026)
- [Epistemic Blinding: An Inference-Time Protocol for Auditing Prior Contamination in LLM-Assisted Analysis](https://mlsanity.com/paper/2604.06013/) (2026)
- [LLMBDC: Language Model for Biological Domains Oriented Clustering of Gene Ontology](https://mlsanity.com/paper/2608.00099/) (2026)
- [SoftGene: Protein Language Model-Enhanced Soft Prompting for Interpretable Gene Set Annotation](https://mlsanity.com/paper/2610.03029/) (2026)

## Abstract (arXiv)

Gene set analysis is a mainstay of functional genomics, but it relies on curated databases of gene functions that are incomplete. Here we evaluate five Large Language Models (LLMs) for their ability to discover the common biological functions represented by a gene set, substantiated by supporting rationale, citations and a confidence assessment. Benchmarking against canonical gene sets from the Gene Ontology, GPT-4 confidently recovered the curated name or a more general concept (73% of cases), while benchmarking against random gene sets correctly yielded zero confidence. Gemini-Pro and Mixtral-Instruct showed ability in naming but were falsely confident for random sets, whereas Llama2-70b had poor performance overall. In gene sets derived from 'omics data, GPT-4 identified novel functions not reported by classical functional enrichment (32% of cases), which independent review indicated were largely verifiable and not hallucinations. The ability to rapidly synthesize common gene functions positions LLMs as valuable 'omics assistants.