# Evaluating Automated Radiology Report Quality through Fine-Grained Phrasal Grounding of Clinical Findings

arXiv: https://arxiv.org/abs/2412.01031 · HTML: https://mlsanity.com/paper/2412.01031/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 10 of 10)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [RadGraph: Extracting Clinical Entities and Relations from Radiology Reports](https://mlsanity.com/paper/2106.14463/) (2021)
- [Interactive and Explainable Region-guided Radiology Report Generation](https://mlsanity.com/paper/2304.08295/) (2023)
- [XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models](https://mlsanity.com/paper/2306.07971/) (2023)
- [Chest X-ray Report Generation through Fine-Grained Label Learning](https://mlsanity.com/paper/2007.13831/) (2020)
- [Automated Generation of Accurate & Fluent Medical X-ray Reports](https://mlsanity.com/paper/2108.12126/) (2021)
- [Extracting and Learning Fine-Grained Labels from Chest Radiographs](https://mlsanity.com/paper/2011.09517/) (2020)
- [Evaluation of GPT-4 for chest X-ray impression generation: A reader study on performance and perception](https://mlsanity.com/paper/2311.06815/) (2023)

## Built on by (most cited first, 3 of 3)

- [MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data](https://mlsanity.com/paper/2601.06847/) (2026)
- [RadSEM: A Finding-by-Finding Metric for Clinical Consistency in Radiology Reports](https://mlsanity.com/paper/2606.17062/) (2026)
- [Phrase-grounded Fact-checking for Automatically Generated Chest X-ray Reports](https://mlsanity.com/paper/2509.21356/) (2025)

## Abstract (arXiv)

Several evaluation metrics have been developed recently to automatically assess the quality of generative AI reports for chest radiographs based only on textual information using lexical, semantic, or clinical named entity recognition methods. In this paper, we develop a new method of report quality evaluation by first extracting fine-grained finding patterns capturing the location, laterality, and severity of a large number of clinical findings. We then performed phrasal grounding to localize their associated anatomical regions on chest radiograph images. The textual and visual measures are then combined to rate the quality of the generated reports. We present results that compare this evaluation metric with other textual metrics on a gold standard dataset derived from the MIMIC collection and show its robustness and sensitivity to factual errors.