# Radiology-GPT: A Large Language Model for Radiology

arXiv: https://arxiv.org/abs/2306.08666 · HTML: https://mlsanity.com/paper/2306.08666/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 57 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4](https://mlsanity.com/paper/2303.11032/) (2023)
- [Interpretability at Scale: Identifying Causal Mechanisms in Alpaca](https://mlsanity.com/paper/2305.08809/) (2023)
- [An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT](https://mlsanity.com/paper/2304.08448/) (2023)
- [When Brain-inspired AI Meets AGI](https://mlsanity.com/paper/2303.15935/) (2023)
- [Context Matters: A Strategy to Pre-train Language Model for Science Education](https://mlsanity.com/paper/2301.12031/) (2023)
- [Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task](https://mlsanity.com/paper/2304.09138/) (2023)
- [ChatABL: Abductive Learning via Natural Language Interaction with ChatGPT](https://mlsanity.com/paper/2304.11107/) (2023)
- [Mask-guided BERT for Few Shot Text Classification](https://mlsanity.com/paper/2302.10447/) (2023)
- [Word Graph Guided Summarization for Radiology Findings](https://mlsanity.com/paper/2112.09925/) (2021)

## Built on by (most cited first, 30 of 44)

- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [Revolutionizing Finance with LLMs: An Overview of Applications and Insights](https://mlsanity.com/paper/2401.11641/) (2024)
- [Advancing Multimodal Medical Capabilities of Gemini](https://mlsanity.com/paper/2405.03162/) (2024)
- [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023)
- [MAIRA-1: A specialised large multimodal model for radiology report generation](https://mlsanity.com/paper/2311.13668/) (2023)
- [Understanding LLMs: A Comprehensive Overview from Training to Inference](https://mlsanity.com/paper/2401.02038/) (2024)
- [Radiology-Llama2: Best-in-Class Large Language Model for Radiology](https://mlsanity.com/paper/2309.06419/) (2023)
- [SAMAug: Point Prompt Augmentation for Segment Anything Model](https://mlsanity.com/paper/2307.01187/) (2023)
- [GeoGalactica: A Scientific Large Language Model in Geoscience](https://mlsanity.com/paper/2401.00434/) (2023)
- [An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation](https://mlsanity.com/paper/2410.03334/) (2024)
- [Segment Anything Model (SAM) for Radiation Oncology](https://mlsanity.com/paper/2306.11730/) (2023)
- [Multimodality of AI for Education: Towards Artificial General Intelligence](https://mlsanity.com/paper/2312.06037/) (2023)
- [Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models](https://mlsanity.com/paper/2311.00287/) (2023)
- [LLM-driven Multimodal Target Volume Contouring in Radiation Oncology](https://mlsanity.com/paper/2311.01908/) (2023)
- [RadOnc-GPT: A Large Language Model for Radiation Oncology](https://mlsanity.com/paper/2309.10160/) (2023)
- [ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data](https://mlsanity.com/paper/2310.05242/) (2023)
- [Evaluating Large Language Models for Radiology Natural Language Processing](https://mlsanity.com/paper/2307.13693/) (2023)
- [Geneverse: A collection of Open-source Multimodal Large Language Models for Genomic and Proteomic Research](https://mlsanity.com/paper/2406.15534/) (2024)
- [Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review](https://mlsanity.com/paper/2311.01918/) (2023)
- [From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practice](https://mlsanity.com/paper/2410.01812/) (2024)
- [Multimodal Foundation Models For Echocardiogram Interpretation](https://mlsanity.com/paper/2308.15670/) (2023)
- [Hierarchical Semantic Tree Concept Whitening for Interpretable Image Classification](https://mlsanity.com/paper/2307.04343/) (2023)
- [GPT vs Human for Scientific Reviews: A Dual Source Review on Applications of ChatGPT in Science](https://mlsanity.com/paper/2312.03769/) (2023)
- [TV-SAM: Increasing Zero-Shot Segmentation Performance on Multimodal Medical Images Using GPT-4 Generated Descriptive Prompts Without Human Annotation](https://mlsanity.com/paper/2402.15759/) (2024)
- [RadOnc-GPT: An Autonomous LLM Agent for Real-Time Patient Outcomes Labeling at Scale](https://mlsanity.com/paper/2509.25540/) (2025)
- [KwaiYiiMath: Technical Report](https://mlsanity.com/paper/2310.07488/) (2023)
- [RAD-PHI2: Instruction Tuning PHI-2 for Radiology](https://mlsanity.com/paper/2403.09725/) (2024)
- [RadPhi-3: Small Language Models for Radiology](https://mlsanity.com/paper/2411.13604/) (2024)
- [Gla-AI4BioMed at RRG24: Visual Instruction-tuned Adaptation for Radiology Report Generation](https://mlsanity.com/paper/2412.04954/) (2024)
- [PharmacyGPT: The AI Pharmacist](https://mlsanity.com/paper/2307.10432/) (2023)

## Abstract (arXiv)

We introduce Radiology-GPT, a large language model for radiology. Using an instruction tuning approach on an extensive dataset of radiology domain knowledge, Radiology-GPT demonstrates superior performance compared to general language models such as StableLM, Dolly and LLaMA. It exhibits significant versatility in radiological diagnosis, research, and communication. This work serves as a catalyst for future developments in clinical NLP. The successful implementation of Radiology-GPT is indicative of the potential of localizing generative large language models, specifically tailored for distinctive medical specialties, while ensuring adherence to privacy standards such as HIPAA. The prospect of developing individualized, large-scale language models that cater to specific needs of various hospitals presents a promising direction. The fusion of conversational competence and domain-specific knowledge in these models is set to foster future development in healthcare AI. A demo of Radiology-GPT is available at https://huggingface.co/spaces/allen-eric/radiology-gpt.