# Evaluation of GPT and BERT-based models on identifying protein-protein interactions in biomedical text

arXiv: https://arxiv.org/abs/2303.17728 · HTML: https://mlsanity.com/paper/2303.17728/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 52 references that are in the map.
- First posted on arXiv 30 Mar 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [Construction of the Literature Graph in Semantic Scholar](https://mlsanity.com/paper/1805.02262/) (2018)
- [Deep learning for extracting protein-protein interactions from biomedical literature](https://mlsanity.com/paper/1706.01556/) (2017)
- [Identifying Protein-Protein Interaction using Tree LSTM and Structured Attention](https://mlsanity.com/paper/1808.03227/) (2018)

## Built on by (most cited first, 9 of 9)

- [Benchmarking large language models for biomedical natural language processing applications and recommendations](https://mlsanity.com/paper/2305.16326/) (2023)
- [Seq vs Seq: An Open Suite of Paired Encoders and Decoders](https://mlsanity.com/paper/2507.11412/) (2025)
- [A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Tasks](https://mlsanity.com/paper/2307.12114/) (2023)
- [Bioinformatics and Biomedical Informatics with ChatGPT: Year One Review](https://mlsanity.com/paper/2403.15274/) (2024)
- [A Comprehensive Analysis of Static Word Embeddings for Turkish](https://mlsanity.com/paper/2405.07778/) (2024)
- [An Eye on Clinical BERT: Investigating Language Model Generalization for Diabetic Eye Disease Phenotyping](https://mlsanity.com/paper/2311.08687/) (2023)
- [Cheap Learning: Maximising Performance of Language Models for Social Data Science Using Minimal Data](https://mlsanity.com/paper/2401.12295/) (2024)
- [Fine-Tuning Pre-trained Language Models to Detect In-Game Trash Talks](https://mlsanity.com/paper/2403.15458/) (2024)
- [PlantDeBERTa: An Open Source Language Model for Plant Science](https://mlsanity.com/paper/2506.08897/) (2025)

## Abstract (arXiv)

Detecting protein-protein interactions (PPIs) is crucial for understanding genetic mechanisms, disease pathogenesis, and drug design. However, with the fast-paced growth of biomedical literature, there is a growing need for automated and accurate extraction of PPIs to facilitate scientific knowledge discovery. Pre-trained language models, such as generative pre-trained transformers (GPT) and bidirectional encoder representations from transformers (BERT), have shown promising results in natural language processing (NLP) tasks. We evaluated the performance of PPI identification of multiple GPT and BERT models using three manually curated gold-standard corpora: Learning Language in Logic (LLL) with 164 PPIs in 77 sentences, Human Protein Reference Database with 163 PPIs in 145 sentences, and Interaction Extraction Performance Assessment with 335 PPIs in 486 sentences. BERT-based models achieved the best overall performance, with BioBERT achieving the highest recall (91.95%) and F1-score (86.84%) and PubMedBERT achieving the highest precision (85.25%). Interestingly, despite not being explicitly trained for biomedical texts, GPT-4 achieved commendable performance, comparable to the top-performing BERT models. It achieved a precision of 88.37%, a recall of 85.14%, and an F1-score of 86.49% on the LLL dataset. These results suggest that GPT models can effectively detect PPIs from text data, offering promising avenues for application in biomedical literature mining. Further research could explore how these models might be fine-tuned for even more specialized tasks within the biomedical domain.