mlsanityData to 14 Oct 2026

ReXrank: A Public Leaderboard for AI-Powered Radiology Report Generation

arXiv 2411.15122 ↗First posted 22 Nov 2024cs.CV cs.AI cs.CLcited by 25

Not on the map. Why?

Abstract (arXiv)

AI-driven models have demonstrated significant potential in automating radiology report generation for chest X-rays. However, there is no standardized benchmark for objectively evaluating their performance. To address this, we present ReXrank, https://rexrank.ai, a public leaderboard and challenge for assessing AI-powered radiology report generation. Our framework incorporates ReXGradient, the largest test dataset consisting of 10,000 studies, and three public datasets (MIMIC-CXR, IU-Xray, CheXpert Plus) for report generation assessment. ReXrank employs 8 evaluation metrics and separately assesses models capable of generating only findings sections and those providing both findings and impressions sections. By providing this standardized evaluation framework, ReXrank enables meaningful comparisons of model performance and offers crucial insights into their robustness across diverse clinical settings. Beyond its current focus on chest X-rays, ReXrank's framework sets the stage for comprehensive evaluation of automated reporting across the full spectrum of medical imaging.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 22 Nov 2024.

Builds on

18 of 30 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  3. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  4. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  5. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  6. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  7. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 337
  8. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 319
  9. 2023BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Taskscited by 201
  10. 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 179
  11. 2024MAIRA-2: Grounded Radiology Report Generationcited by 145
  12. 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 140
6 more
  1. 2024GREEN: Generative Radiology Report Evaluation and Error Notationcited by 128
  2. 2022Improving Chest X-Ray Report Generation by Leveraging Warm Startingcited by 95
  3. 2021Chest ImaGenome Dataset for Clinical Reasoningcited by 90
  4. 2024RaTEScore: A Metric for Radiology Report Generationcited by 65
  5. 2023LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generationcited by 57
  6. 2024FineRadScore: A Radiology Report Line-by-Line Evaluation Technique Generating Corrections with Severity Scorescited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 13.

25 papers.

Sort:

By area:

  1. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 230
  2. 2025ReXGradient-160K: A Large-Scale Publicly Available Dataset of Chest Radiographs with Free-text Reportscited by 27
  3. 2025RadVLM: A Multitask Conversational Vision-Language Model for Radiologycited by 25
  4. 2025ReXGroundingCT: A 3D Chest CT Dataset for Segmentation of Findings from Free-Text Reportscited by 17
  5. 2026Scaling medical imaging report generation with multimodal reinforcement learningcited by 9
  6. 2025DDaTR: Dynamic Difference-aware Temporal Residual Network for Longitudinal Radiology Report Generationcited by 8
  7. 2025From large language models to multimodal AI: A scoping review on the potential of generative AI in medicinecited by 5
  8. 2025CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographscited by 5
  9. 2026A Reasoning-Enabled Vision-Language Foundation Model for Chest X-ray Interpretationcited by 5
  10. 2026LiveClin: A Live Clinical Benchmark without Leakagecited by 4
  11. 2026Discrete Diffusion Language Models for Interactive Radiology Report Draftingcited by 2
  12. 2025CorBenchX: Large-Scale Chest X-Ray Error Dataset and Vision-Language Model Benchmark for Report Error Correctioncited by 1
13 more
  1. 2025Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rayscited by 1
  2. 2025Random Direct Preference Optimization for Radiography Report Generationcited by 1
  3. 2026MI-CXR: A Benchmark for Longitudinal Reasoning over Multi-Interval Chest X-rayscited by 1
  4. 2025RadHiera: Semantic Hierarchical Reinforcement Learning for Medical Report Generationcited by 0
  5. 2025Benchmark Success, Clinical Failure: When Reinforcement Learning Optimizes for Benchmarks, Not Patientscited by 0
  6. 2026Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotationcited by 0
  7. 2026Ran Score: a LLM-based Evaluation Score for Radiology Report Generationcited by 0
  8. 2026ReXSonoVQA: A Video QA Benchmark for Procedure-Centric Ultrasound Understandingcited by 0
  9. 2026CXR-ContraBench: Benchmarking Negated-Option Attraction in Medical VLMscited by 0
  10. 2026SHOVIR: A Benchmark for Evaluating Vision Shortcut Learning in Radiology Report Generationcited by 0
  11. 2026CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurementcited by 0
  12. 2026Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluationcited by 0
  13. 2026BrainTRACE: Tracing Longitudinal, Multimodal, and Volumetric Evidence in Brain MRI Clinical Reasoningcited by 0