# Protocol for evaluating ChatGPT in biomedical association generation and verification using a RAG-enabled, cross-model majority voting workflow

arXiv: https://arxiv.org/abs/2605.30400 · HTML: https://mlsanity.com/paper/2605.30400/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 29 references that are in the map.
- First posted on arXiv 28 May 2026.

## Builds on (most cited first, 7 of 7)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [The Faiss library](https://mlsanity.com/paper/2401.08281/) (2024)
- [Shifting Polarization and Twitter News Influencers between two U.S. Presidential Elections](https://mlsanity.com/paper/2111.02505/) (2021)
- [From Knowledge Generation to Knowledge Verification: Examining the BioMedical Generative Capabilities of ChatGPT](https://mlsanity.com/paper/2502.14714/) (2025)
- [Fact-Checking Generative AI: Ontology-Driven Biological Graphs for Disease-Gene Link Verification](https://mlsanity.com/paper/2308.03929/) (2023)
- [Accelerating Complex Disease Treatment through Network Medicine and GenAI: A Case Study on Drug Repurposing for Breast Cancer](https://mlsanity.com/paper/2406.13106/) (2024)

## Abstract (arXiv)

We present a protocol to evaluate ChatGPT's ability to generate disease-centric biomedical associations. It outlines how we generate the associations, validate the biological entities using biomedical ontologies, and verify associations using literature. The protocol includes a self-consistency strategy to assess generative reliability across ChatGPT models. To address ontology exact-match limitations, we provide a use case performing semantic verification through a workflow enabled by Retrieval-Augmented Generation (RAG) powered by open-source large language models (LLMs). This enables LLMs to establish truth over content generated by other LLMs and expose hallucination.