# CancerLLM: A Large Language Model in Cancer Domain

arXiv: https://arxiv.org/abs/2406.10459 · HTML: https://mlsanity.com/paper/2406.10459/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 52 references that are in the map.
- First posted on arXiv 15 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains](https://mlsanity.com/paper/2402.10373/) (2024)
- [MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data](https://mlsanity.com/paper/2304.08247/) (2023)
- [ToolQA: A Dataset for LLM Question Answering with External Tools](https://mlsanity.com/paper/2306.13304/) (2023)
- [SGPT: GPT Sentence Embeddings for Semantic Search](https://mlsanity.com/paper/2202.08904/) (2022)
- [MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval](https://mlsanity.com/paper/2307.00589/) (2023)
- [GPT-NER: Named Entity Recognition via Large Language Models](https://mlsanity.com/paper/2304.10428/) (2023)
- [SciRepEval: A Multi-Format Benchmark for Scientific Document Representations](https://mlsanity.com/paper/2211.13308/) (2022)
- [Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding](https://mlsanity.com/paper/2305.12031/) (2023)
- [Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training](https://mlsanity.com/paper/2306.03166/) (2023)
- [Revisiting Summarization Evaluation for Scientific Articles](https://mlsanity.com/paper/1604.00400/) (2016)
- [BiomedRAG: A Retrieval Augmented Large Language Model for Biomedicine](https://mlsanity.com/paper/2405.00465/) (2024)
- [How far is Language Model from 100% Few-shot Named Entity Recognition in Medical Domain](https://mlsanity.com/paper/2307.00186/) (2023)
- [A Hierarchical N-Gram Framework for Zero-Shot Link Prediction](https://mlsanity.com/paper/2204.10293/) (2022)
- [Understand the Dynamic World: An End-to-End Knowledge Informed Framework for Open Domain Entity State Tracking](https://mlsanity.com/paper/2304.13854/) (2023)
- [Enhancing Question Answering for Enterprise Knowledge Bases using Large Language Models](https://mlsanity.com/paper/2404.08695/) (2024)
- [A Condensed Transition Graph Framework for Zero-shot Link Prediction with Large Language Models](https://mlsanity.com/paper/2402.10779/) (2024)

## Built on by (most cited first, 12 of 12)

- [The Rise of Small Language Models in Healthcare: A Comprehensive Survey](https://mlsanity.com/paper/2504.17119/) (2025)
- [EHRStruct: A Comprehensive Benchmark Framework for Evaluating Large Language Models on Structured Electronic Health Record Tasks](https://mlsanity.com/paper/2511.08206/) (2025)
- [Advanced Deep Learning and Large Language Models: Comprehensive Insights for Cancer Detection](https://mlsanity.com/paper/2504.13186/) (2025)
- [CliCARE: Grounding Large Language Models in Clinical Guidelines for Decision Support over Longitudinal Cancer Electronic Health Records](https://mlsanity.com/paper/2507.22533/) (2025)
- [Generative AI for Healthcare: Fundamentals, Challenges, and Perspectives](https://mlsanity.com/paper/2510.24551/) (2025)
- [Automatic Prompt Optimization for Dataset-Level Feature Discovery](https://mlsanity.com/paper/2601.13922/) (2026)
- [GeneSUM: Large Language Model-based Gene Summary Extraction](https://mlsanity.com/paper/2412.18154/) (2024)
- [A Language Vision Model Approach for Automated Tumor Contouring in Radiation Oncology](https://mlsanity.com/paper/2503.14933/) (2025)
- [Efficient and Effective Internal Memory Retrieval for LLM-Based Healthcare Prediction](https://mlsanity.com/paper/2604.07659/) (2026)
- [Evaluating Small Language Models for Agentic On-Farm Decision Support Systems](https://mlsanity.com/paper/2512.14043/) (2025)
- [A collective capability boundary in frontier large language models on guideline-conformant and case-specific oncology decision-making](https://mlsanity.com/paper/2608.28592/) (2026)
- [RIVO: Lightweight Adaptive Rerouting for Deep Research Agents](https://mlsanity.com/paper/2610.15910/) (2026)

## Abstract (arXiv)

Medical Large Language Models (LLMs) have demonstrated impressive performance on a wide variety of medical NLP tasks; however, there still lacks a LLM specifically designed for phenotyping identification and diagnosis in cancer domain. Moreover, these LLMs typically have several billions of parameters, making them computationally expensive for healthcare systems. Thus, in this study, we propose CancerLLM, a model with 7 billion parameters and a Mistral-style architecture, pre-trained on nearly 2.7M clinical notes and over 515K pathology reports covering 17 cancer types, followed by fine-tuning on two cancer-relevant tasks, including cancer phenotypes extraction and cancer diagnosis generation. Our evaluation demonstrated that the CancerLLM achieves state-of-the-art results with F1 score of 91.78% on phenotyping extraction and 86.81% on disganois generation. It outperformed existing LLMs, with an average F1 score improvement of 9.23%. Additionally, the CancerLLM demonstrated its efficiency on time and GPU usage, and robustness comparing with other LLMs. We demonstrated that CancerLLM can potentially provide an effective and robust solution to advance clinical research and practice in cancer domain