# Zero-shot information extraction from radiological reports using ChatGPT

arXiv: https://arxiv.org/abs/2309.01398 · HTML: https://mlsanity.com/paper/2309.01398/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 32 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis](https://mlsanity.com/paper/1706.03446/) (2017)
- [Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition](https://mlsanity.com/paper/1706.09569/) (2017)
- [A frame semantic overview of NLP-based information extraction for cancer-related EHR notes](https://mlsanity.com/paper/1904.01655/) (2019)

## Built on by (most cited first, 15 of 15)

- [LLMs in Biomedicine: A study on clinical Named Entity Recognition](https://mlsanity.com/paper/2404.07376/) (2024)
- [Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysis](https://mlsanity.com/paper/2403.16303/) (2024)
- [The current status of large language models in summarizing radiology report impressions](https://mlsanity.com/paper/2406.02134/) (2024)
- [Entity Decomposition with Filtering: A Zero-Shot Clinical Named Entity Recognition Framework](https://mlsanity.com/paper/2407.04629/) (2024)
- [To what extent is ChatGPT useful for language teacher lesson plan creation?](https://mlsanity.com/paper/2407.09974/) (2024)
- [Effectiveness of ChatGPT in explaining complex medical reports to patients](https://mlsanity.com/paper/2406.15963/) (2024)
- [CACER: Clinical Concept Annotations for Cancer Events and Relations](https://mlsanity.com/paper/2409.03905/) (2024)
- [Evaluation of General Large Language Models in Contextually Assessing Semantic Concepts Extracted from Adult Critical Care Electronic Health Record Notes](https://mlsanity.com/paper/2401.13588/) (2024)
- [The Power of Combining Data and Knowledge: GPT-4o is an Effective Interpreter of Machine Learning Models in Predicting Lymph Node Metastasis of Lung Cancer](https://mlsanity.com/paper/2407.17900/) (2024)
- [Advancing Chinese biomedical text mining with community challenges](https://mlsanity.com/paper/2403.04261/) (2024)
- [Natural Language Processing for Analyzing Electronic Health Records and Clinical Notes in Cancer Research: A Review](https://mlsanity.com/paper/2410.22180/) (2024)
- [Can ChatGPT Diagnose Alzheimer's Disease?](https://mlsanity.com/paper/2502.06907/) (2025)
- [Enhancing LLMs for Identifying and Prioritizing Important Medical Jargons from Electronic Health Record Notes Utilizing Data Augmentation: A Comparative Study](https://mlsanity.com/paper/2502.16022/) (2025)
- [Review GIDE -- Restaurant Review Gastrointestinal Illness Detection and Extraction with Large Language Models](https://mlsanity.com/paper/2503.09743/) (2025)
- [RadAnnotate: Large Language Models for Efficient and Reliable Radiology Report Annotation](https://mlsanity.com/paper/2603.16002/) (2026)

## Abstract (arXiv)

Electronic health records contain an enormous amount of valuable information, but many are recorded in free text. Information extraction is the strategy to transform the sequence of characters into structured data, which can be employed for secondary analysis. However, the traditional information extraction components, such as named entity recognition and relation extraction, require annotated data to optimize the model parameters, which has become one of the major bottlenecks in building information extraction systems. With the large language models achieving good performances on various downstream NLP tasks without parameter tuning, it becomes possible to use large language models for zero-shot information extraction. In this study, we aim to explore whether the most popular large language model, ChatGPT, can extract useful information from the radiological reports. We first design the prompt template for the interested information in the CT reports. Then, we generate the prompts by combining the prompt template with the CT reports as the inputs of ChatGPT to obtain the responses. A post-processing module is developed to transform the responses into structured extraction results. We conducted the experiments with 847 CT reports collected from Peking University Cancer Hospital. The experimental results indicate that ChatGPT can achieve competitive performances for some extraction tasks compared with the baseline information extraction system, but some limitations need to be further improved.