# Understanding Spatial Language in Radiology: Representation Framework, Annotation, and Spatial Relation Extraction from Chest X-ray Reports using Deep Learning

arXiv: https://arxiv.org/abs/1908.04485 · HTML: https://mlsanity.com/paper/1908.04485/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2019.

## Built on by (most cited first, 12 of 12)

- [Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing](https://mlsanity.com/paper/2204.09817/) (2022)
- [RadGraph: Extracting Clinical Entities and Relations from Radiology Reports](https://mlsanity.com/paper/2106.14463/) (2021)
- [Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning](https://mlsanity.com/paper/2210.16952/) (2022)
- [Your other Left! Vision-Language Models Fail to Identify Relative Positions in Medical Images](https://mlsanity.com/paper/2508.00549/) (2025)
- [RadGraph2: Modeling Disease Progression in Radiology Reports via Hierarchical Information Extraction](https://mlsanity.com/paper/2308.05046/) (2023)
- [Event-based clinical findings extraction from radiology reports with pre-trained language model](https://mlsanity.com/paper/2112.13512/) (2021)
- [DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Text](https://mlsanity.com/paper/2310.12557/) (2023)
- [A Novel Corpus of Annotated Medical Imaging Reports and Information Extraction Results Using BERT-based Language Models](https://mlsanity.com/paper/2403.18975/) (2024)
- [Anatomy-VLM: A Fine-grained Vision-Language Model for Medical Interpretation](https://mlsanity.com/paper/2511.08402/) (2025)
- [VLM-KG: Multimodal Radiology Knowledge Graph Generation](https://mlsanity.com/paper/2505.17042/) (2025)
- [Identifying ARDS using the Hierarchical Attention Network with Sentence Objectives Framework](https://mlsanity.com/paper/2103.06352/) (2021)
- [A Modular Agent for Reliable and Auditable Spatial Relation Verification in CT Scans](https://mlsanity.com/paper/2608.21140/) (2026)

## Abstract (arXiv)

We define a representation framework for extracting spatial information from radiology reports (Rad-SpRL). We annotated a total of 2000 chest X-ray reports with 4 spatial roles corresponding to the common radiology entities. Our focus is on extracting detailed information of a radiologist's interpretation containing a radiographic finding, its anatomical location, corresponding probable diagnoses, as well as associated hedging terms. For this, we propose a deep learning-based natural language processing (NLP) method involving both word and character-level encodings. Specifically, we utilize a bidirectional long short-term memory (Bi-LSTM) conditional random field (CRF) model for extracting the spatial roles. The model achieved average F1 measures of 90.28 and 94.61 for extracting the Trajector and Landmark roles respectively whereas the performance was moderate for Diagnosis and Hedge roles with average F1 of 71.47 and 73.27 respectively. The corpus will soon be made available upon request.