# Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting

arXiv: https://arxiv.org/abs/2307.05766 · HTML: https://mlsanity.com/paper/2307.05766/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 11 Jul 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Pix2seq: A Language Modeling Framework for Object Detection](https://mlsanity.com/paper/2109.10852/) (2021)
- [Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training](https://mlsanity.com/paper/2209.07098/) (2022)
- [Overcoming Data Limitation in Medical Visual Question Answering](https://mlsanity.com/paper/1909.11867/) (2019)
- [MMBERT: Multimodal BERT Pretraining for Improved Medical VQA](https://mlsanity.com/paper/2104.01394/) (2021)
- [Multiple Meta-model Quantifying for Medical Visual Question Answering](https://mlsanity.com/paper/2105.08913/) (2021)
- [A Medical Semantic-Assisted Transformer for Radiographic Report Generation](https://mlsanity.com/paper/2208.10358/) (2022)
- [RATCHET: Medical Transformer for Chest X-ray Diagnosis and Reporting](https://mlsanity.com/paper/2107.02104/) (2021)
- [Chest X-ray Report Generation through Fine-Grained Label Learning](https://mlsanity.com/paper/2007.13831/) (2020)
- [Improving Pneumonia Localization via Cross-Attention on Medical Images and Reports](https://mlsanity.com/paper/2110.03094/) (2021)
- [A Self-Guided Framework for Radiology Report Generation](https://mlsanity.com/paper/2206.09378/) (2022)
- [RepsNet: Combining Vision with Language for Automated Medical Reports](https://mlsanity.com/paper/2209.13171/) (2022)
- [Consistency-preserving Visual Question Answering in Medical Imaging](https://mlsanity.com/paper/2206.13296/) (2022)

## Built on by (most cited first, 18 of 18)

- [MedRAX: Medical Reasoning Agent for Chest X-ray](https://mlsanity.com/paper/2502.02673/) (2025)
- [A Systematic Evaluation of GPT-4V's Multimodal Capability for Medical Image Analysis](https://mlsanity.com/paper/2310.20381/) (2023)
- [RadAgents: Multimodal Agentic Reasoning for Chest X-ray Interpretation with Radiologist-like Workflows](https://mlsanity.com/paper/2509.20490/) (2025)
- [It is Too Many Options: Pitfalls of Multiple-Choice Questions in Generative AI and Medical Education](https://mlsanity.com/paper/2503.13508/) (2025)
- [A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data](https://mlsanity.com/paper/2405.12833/) (2024)
- [S-RRG-Bench: Structured Radiology Report Generation with Fine-Grained Evaluation Framework](https://mlsanity.com/paper/2508.02082/) (2025)
- [From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine](https://mlsanity.com/paper/2502.09242/) (2025)
- [XrayClaw: Cooperative-Competitive Multi-Agent Alignment for Trustworthy Chest X-ray Diagnosis](https://mlsanity.com/paper/2604.02695/) (2026)
- [CheXalign: Preference fine-tuning in chest X-ray interpretation models without human feedback](https://mlsanity.com/paper/2410.07025/) (2024)
- [Hierarchical Modeling for Medical Visual Question Answering with Cross-Attention Fusion](https://mlsanity.com/paper/2504.03135/) (2025)
- [RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistance](https://mlsanity.com/paper/2311.18681/) (2023)
- [Efficient Bilinear Attention-based Fusion for Medical Visual Question Answering](https://mlsanity.com/paper/2410.21000/) (2024)
- [Interpreting Chest X-rays Like a Radiologist: A Benchmark with Clinical Reasoning](https://mlsanity.com/paper/2505.23143/) (2025)
- [Prototype-Based Knowledge Guidance for Fine-Grained Structured Radiology Reporting](https://mlsanity.com/paper/2603.11938/) (2026)
- [Calibrated Confidence Expression for Radiology Report Generation](https://mlsanity.com/paper/2603.29492/) (2026)
- [AGA3DNet: Anatomy-Guided Gaussian Priors with Multi-view xLSTM for 3D Brain MRI Subtype Classification](https://mlsanity.com/paper/2605.07142/) (2026)
- [RadFusion: Towards Threshold-Controllable Radiology Report Generation](https://mlsanity.com/paper/2608.10505/) (2026)
- [X-Ragent: Tool-Augmented Reinforcement Learning for Evidence-Grounded Chest X-Ray Reasoning](https://mlsanity.com/paper/2610.16811/) (2026)

## Abstract (arXiv)

Radiology reporting is a crucial part of the communication between radiologists and other medical professionals, but it can be time-consuming and error-prone. One approach to alleviate this is structured reporting, which saves time and enables a more accurate evaluation than free-text reports. However, there is limited research on automating structured reporting, and no public benchmark is available for evaluating and comparing different methods. To close this gap, we introduce Rad-ReStruct, a new benchmark dataset that provides fine-grained, hierarchically ordered annotations in the form of structured reports for X-Ray images. We model the structured reporting task as hierarchical visual question answering (VQA) and propose hi-VQA, a novel method that considers prior context in the form of previously asked questions and answers for populating a structured radiology report. Our experiments show that hi-VQA achieves competitive performance to the state-of-the-art on the medical VQA benchmark VQARad while performing best among methods without domain-specific vision-language pretraining and provides a strong baseline on Rad-ReStruct. Our work represents a significant step towards the automated population of structured radiology reports and provides a valuable first benchmark for future research in this area. Our dataset and code is available at https://github.com/ChantalMP/Rad-ReStruct.