# CancerGPT: Few-shot Drug Pair Synergy Prediction using Large Pre-trained Language Models

arXiv: https://arxiv.org/abs/2304.10946 · HTML: https://mlsanity.com/paper/2304.10946/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 48 references that are in the map.
- First posted on arXiv 18 Apr 2023.

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [TabLLM: Few-shot Classification of Tabular Data with Large Language Models](https://mlsanity.com/paper/2210.10723/) (2022)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [Can Foundation Models Wrangle Your Data?](https://mlsanity.com/paper/2205.09911/) (2022)
- [Deep Entity Matching with Pre-Trained Language Models](https://mlsanity.com/paper/2004.00584/) (2020)
- [The Debate Over Understanding in AI's Large Language Models](https://mlsanity.com/paper/2210.13966/) (2022)
- [Learning From Noisy Large-Scale Datasets With Minimal Supervision](https://mlsanity.com/paper/1701.01619/) (2017)
- [STUNT: Few-shot Tabular Learning with Self-generated Tasks from Unlabeled Tables](https://mlsanity.com/paper/2303.00918/) (2023)
- [Few-Shot Learning with Localization in Realistic Settings](https://mlsanity.com/paper/1904.08502/) (2019)

## Built on by (most cited first, 11 of 11)

- [Instruction Backdoor Attacks Against Customized LLMs](https://mlsanity.com/paper/2402.09179/) (2024)
- [Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation](https://mlsanity.com/paper/2305.00909/) (2023)
- [EEG-GPT: Exploring Capabilities of Large Language Models for EEG Classification and Interpretation](https://mlsanity.com/paper/2401.18006/) (2024)
- [MarineGPT: Unlocking Secrets of Ocean to the Public](https://mlsanity.com/paper/2310.13596/) (2023)
- [Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models](https://mlsanity.com/paper/2306.10460/) (2023)
- [SynerGPT: In-Context Learning for Personalized Drug Synergy Prediction and Drug Design](https://mlsanity.com/paper/2307.11694/) (2023)
- [BioKGBench: A Knowledge Graph Checking Benchmark of AI Agent for Biomedical Science](https://mlsanity.com/paper/2407.00466/) (2024)
- [Are Large Kernels Better Teachers than Transformers for ConvNets?](https://mlsanity.com/paper/2305.19412/) (2023)
- [GPT vs Human for Scientific Reviews: A Dual Source Review on Applications of ChatGPT in Science](https://mlsanity.com/paper/2312.03769/) (2023)
- [Llamarine: Open-source Maritime Industry-specific Large Language Model](https://mlsanity.com/paper/2503.00203/) (2025)
- [Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication](https://mlsanity.com/paper/2306.10466/) (2023)

## Abstract (arXiv)

Large pre-trained language models (LLMs) have been shown to have significant potential in few-shot learning across various fields, even with minimal training data. However, their ability to generalize to unseen tasks in more complex fields, such as biology, has yet to be fully evaluated. LLMs can offer a promising alternative approach for biological inference, particularly in cases where structured data and sample size are limited, by extracting prior knowledge from text corpora. Our proposed few-shot learning approach uses LLMs to predict the synergy of drug pairs in rare tissues that lack structured data and features. Our experiments, which involved seven rare tissues from different cancer types, demonstrated that the LLM-based prediction model achieved significant accuracy with very few or zero samples. Our proposed model, the CancerGPT (with ~ 124M parameters), was even comparable to the larger fine-tuned GPT-3 model (with ~ 175B parameters). Our research is the first to tackle drug pair synergy prediction in rare tissues with limited data. We are also the first to utilize an LLM-based prediction model for biological reaction prediction tasks.