# Uncovering Knowledge Gaps in Radiology Report Generation Models through Knowledge Graphs

arXiv: https://arxiv.org/abs/2408.14397 · HTML: https://mlsanity.com/paper/2408.14397/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Builds on (most cited first, 27 of 27)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT](https://mlsanity.com/paper/2004.09167/) (2020)
- [RadGraph: Extracting Clinical Entities and Relations from Radiology Reports](https://mlsanity.com/paper/2106.14463/) (2021)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval](https://mlsanity.com/paper/2307.00589/) (2023)
- [When Radiology Report Generation Meets Knowledge Graph](https://mlsanity.com/paper/2002.08277/) (2020)
- [MAIRA-2: Grounded Radiology Report Generation](https://mlsanity.com/paper/2406.04449/) (2024)
- [Interactive and Explainable Region-guided Radiology Report Generation](https://mlsanity.com/paper/2304.08295/) (2023)
- [GREEN: Generative Radiology Report Evaluation and Error Notation](https://mlsanity.com/paper/2405.03595/) (2024)
- [Improving Chest X-Ray Report Generation by Leveraging Warm Starting](https://mlsanity.com/paper/2201.09405/) (2022)
- [Chest ImaGenome Dataset for Clinical Reasoning](https://mlsanity.com/paper/2108.00316/) (2021)
- [A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation](https://mlsanity.com/paper/2401.12208/) (2024)
- [NegBio: a high-performance tool for negation and uncertainty detection in radiology reports](https://mlsanity.com/paper/1712.05898/) (2017)
- [RaTEScore: A Metric for Radiology Report Generation](https://mlsanity.com/paper/2406.16845/) (2024)
- [Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation](https://mlsanity.com/paper/2403.08002/) (2024)
- [Relation Extraction : A Survey](https://mlsanity.com/paper/1712.05191/) (2017)
- [MedVersa: A Generalist Foundation Model for Medical Image Interpretation](https://mlsanity.com/paper/2405.07988/) (2024)
- [CheXpert++: Approximating the CheXpert labeler for Speed,Differentiability, and Probabilistic Output](https://mlsanity.com/paper/2006.15229/) (2020)
- [FineRadScore: A Radiology Report Line-by-Line Evaluation Technique Generating Corrections with Severity Scores](https://mlsanity.com/paper/2405.20613/) (2024)
- [A Systematic Review of Deep Learning-based Research on Radiology Report Generation](https://mlsanity.com/paper/2311.14199/) (2023)
- [RadGraph2: Modeling Disease Progression in Radiology Reports via Hierarchical Information Extraction](https://mlsanity.com/paper/2308.05046/) (2023)
- [Knowledge Graph Construction and Its Application in Automatic Radiology Report Generation from Radiologist's Dictation](https://mlsanity.com/paper/2206.06308/) (2022)

## Built on by (most cited first, 7 of 7)

- [FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Models](https://mlsanity.com/paper/2411.18672/) (2024)
- [R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation](https://mlsanity.com/paper/2508.03426/) (2025)
- [Foundation Models in Biomedical Imaging: Turning Hype into Reality](https://mlsanity.com/paper/2512.15808/) (2025)
- [Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors](https://mlsanity.com/paper/2606.17213/) (2026)
- [Holistic Artificial Intelligence in Medicine; improved performance and explainability](https://mlsanity.com/paper/2507.00205/) (2025)
- [CheXPO-v2: Preference Optimization for Chest X-ray VLMs with Knowledge Graph Consistency](https://mlsanity.com/paper/2512.17213/) (2025)
- [From Image Interpretation to Clinical Reasoning: Upstream Physician-Context-Aware Multimodal Learning with Causal Reinforcement Learning](https://mlsanity.com/paper/2609.38924/) (2026)

## Abstract (arXiv)

Recent advancements in artificial intelligence have significantly improved the automatic generation of radiology reports. However, existing evaluation methods fail to reveal the models' understanding of radiological images and their capacity to achieve human-level granularity in descriptions. To bridge this gap, we introduce a system, named ReXKG, which extracts structured information from processed reports to construct a comprehensive radiology knowledge graph. We then propose three metrics to evaluate the similarity of nodes (ReXKG-NSC), distribution of edges (ReXKG-AMS), and coverage of subgraphs (ReXKG-SCS) across various knowledge graphs. We conduct an in-depth comparative analysis of AI-generated and human-written radiology reports, assessing the performance of both specialist and generalist models. Our study provides a deeper understanding of the capabilities and limitations of current AI models in radiology report generation, offering valuable insights for improving model performance and clinical applicability.