# Prototype-Based Knowledge Guidance for Fine-Grained Structured Radiology Reporting

arXiv: https://arxiv.org/abs/2603.11938 · HTML: https://mlsanity.com/paper/2603.11938/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Builds on (most cited first, 18 of 18)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [MAIRA-2: Grounded Radiology Report Generation](https://mlsanity.com/paper/2406.04449/) (2024)
- [Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignment](https://mlsanity.com/paper/2112.15011/) (2021)
- [Cross-modal Prototype Driven Network for Radiology Report Generation](https://mlsanity.com/paper/2207.04818/) (2022)
- [CXR-LLAVA: a multimodal large language model for interpreting chest X-ray images](https://mlsanity.com/paper/2310.18341/) (2023)
- [Utilizing Longitudinal Chest X-Rays and Reports to Pre-Fill Radiology Reports](https://mlsanity.com/paper/2306.08749/) (2023)
- [CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Dataset](https://mlsanity.com/paper/2410.00379/) (2024)
- [Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation](https://mlsanity.com/paper/2407.15268/) (2024)
- [Improving Pneumonia Localization via Cross-Attention on Medical Images and Reports](https://mlsanity.com/paper/2110.03094/) (2021)
- [Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting](https://mlsanity.com/paper/2307.05766/) (2023)
- [Radiology Report Generation via Multi-objective Preference Optimization](https://mlsanity.com/paper/2412.08901/) (2024)
- [Automated Structured Radiology Report Generation](https://mlsanity.com/paper/2505.24223/) (2025)
- [Replace and Report: NLP Assisted Radiology Report Generation](https://mlsanity.com/paper/2306.17180/) (2023)
- [Historical Report Guided Bi-modal Concurrent Learning for Pathology Report Generation](https://mlsanity.com/paper/2506.18658/) (2025)
- [RadIR: A Scalable Framework for Multi-Grained Medical Image Retrieval via Radiology Report Mining](https://mlsanity.com/paper/2503.04653/) (2025)
- [Hierarchical Modeling for Medical Visual Question Answering with Cross-Attention Fusion](https://mlsanity.com/paper/2504.03135/) (2025)

## Abstract (arXiv)

Structured radiology reporting promises faster, more consistent communication than free text, but automation remains difficult as models must make many fine-grained, discrete decisions about rare findings and attributes from limited structured supervision. In contrast, free-text reports are produced at scale in routine care and implicitly encode fine-grained, image-linked information through detailed descriptions. To leverage this unstructured knowledge, we propose ProtoSR, an approach for injecting free-text information into structured report population. First, we introduce an automatic extraction pipeline that uses an instruction-tuned LLM to mine 80k+ MIMIC-CXR studies and build a multimodal knowledge base aligned with a structured reporting template, representing each answer option with a visual prototype. Using this knowledge base, ProtoSR is trained to retrieve prototypes relevant for the current image-question pair and augment the model predictions through a prototype-conditioned residual, providing a data-driven second opinion that selectively corrects predictions. On the Rad-ReStruct benchmark, ProtoSR achieves state-of-the-art results, with the largest improvements on detailed attribute questions, demonstrating the value of integrating free-text derived signal for fine-grained image understanding.