# Event-based clinical findings extraction from radiology reports with pre-trained language model

arXiv: https://arxiv.org/abs/2112.13512 · HTML: https://mlsanity.com/paper/2112.13512/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 53 references that are in the map.
- First posted on arXiv 27 Dec 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [Joint Modeling of Chest Radiographs and Radiology Reports for Pulmonary Edema Assessment](https://mlsanity.com/paper/2008.09884/) (2020)
- [A Frustratingly Easy Approach for Entity and Relation Extraction](https://mlsanity.com/paper/2010.12812/) (2020)
- [Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path](https://mlsanity.com/paper/1508.03720/) (2015)
- [Annotating Social Determinants of Health Using Active Learning, and Characterizing Determinants Using Neural Event Extraction](https://mlsanity.com/paper/2004.05438/) (2020)
- [NeuroNER: an easy-to-use program for named-entity recognition based on neural networks](https://mlsanity.com/paper/1705.05487/) (2017)
- [Predicting COVID-19 Pneumonia Severity on Chest X-ray with Deep Learning](https://mlsanity.com/paper/2005.11856/) (2020)
- [Understanding Spatial Language in Radiology: Representation Framework, Annotation, and Spatial Relation Extraction from Chest X-ray Reports using Deep Learning](https://mlsanity.com/paper/1908.04485/) (2019)
- [Modelling Radiological Language with Bidirectional Long Short-Term Memory Networks](https://mlsanity.com/paper/1609.08409/) (2016)
- [A Dependency-Based Neural Network for Relation Classification](https://mlsanity.com/paper/1507.04646/) (2015)
- [Extraction and Analysis of Clinically Important Follow-up Recommendations in a Large Radiology Dataset](https://mlsanity.com/paper/1905.05877/) (2019)

## Built on by (most cited first, 3 of 3)

- [Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPT](https://mlsanity.com/paper/2304.04920/) (2023)
- [A Novel Corpus of Annotated Medical Imaging Reports and Information Extraction Results Using BERT-based Language Models](https://mlsanity.com/paper/2403.18975/) (2024)
- [MedicalBench: Evaluating Large Language Models Toward Improved Medical Concept Extraction](https://mlsanity.com/paper/2605.20197/) (2026)

## Abstract (arXiv)

Radiology reports contain a diverse and rich set of clinical abnormalities documented by radiologists during their interpretation of the images. Comprehensive semantic representations of radiological findings would enable a wide range of secondary use applications to support diagnosis, triage, outcomes prediction, and clinical research. In this paper, we present a new corpus of radiology reports annotated with clinical findings. Our annotation schema captures detailed representations of pathologic findings that are observable on imaging ("lesions") and other types of clinical problems ("medical problems"). The schema used an event-based representation to capture fine-grained details, including assertion, anatomy, characteristics, size, count, etc. Our gold standard corpus contained a total of 500 annotated computed tomography (CT) reports. We extracted triggers and argument entities using two state-of-the-art deep learning architectures, including BERT. We then predicted the linkages between trigger and argument entities (referred to as argument roles) using a BERT-based relation extraction model. We achieved the best extraction performance using a BERT model pre-trained on 3 million radiology reports from our institution: 90.9%-93.4% F1 for finding triggers 72.0%-85.6% F1 for arguments roles. To assess model generalizability, we used an external validation set randomly sampled from the MIMIC Chest X-ray (MIMIC-CXR) database. The extraction performance on this validation set was 95.6% for finding triggers and 79.1%-89.7% for argument roles, demonstrating that the model generalized well to the cross-institutional data with a different imaging modality. We extracted the finding events from all the radiology reports in the MIMIC-CXR database and provided the extractions to the research community.