mlsanityData to 7 Oct 2026

GREEN: Generative Radiology Report Evaluation and Error Notation

arXiv 2405.03595 ↗First posted 6 May 2024cs.CL cs.AIcited by 124

Not on the map. Why?

Cited by 41 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Evaluating radiology reports is a challenging problem as factual correctness is extremely important due to the need for accurate medical communication about medical images. Existing automatic evaluation metrics either suffer from failing to consider factual correctness (e.g., BLEU and ROUGE) or are limited in their interpretability (e.g., F1CheXpert and F1RadGraph). In this paper, we introduce GREEN (Generative Radiology Report Evaluation and Error Notation), a radiology report generation metric that leverages the natural language understanding of language models to identify and explain clinically significant errors in candidate reports, both quantitatively and qualitatively. Compared to current metrics, GREEN offers: 1) a score aligned with expert preferences, 2) human interpretable explanations of clinically significant errors, enabling feedback loops with end-users, and 3) a lightweight open-source method that reaches the performance of commercial counterparts. We validate our GREEN metric by comparing it to GPT-4, as well as to error counts of 6 experts and preferences of 2 experts. Our method demonstrates not only higher correlation with expert error counts, but simultaneously higher alignment with expert preferences when compared to previous approaches.

Key facts. Cited by 124 papers in the map (data through 7 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 6 May 2024.

Builds on

18 of 35 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  6. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  7. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 329
  8. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
  9. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 308
  10. 2023Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generationcited by 114
  11. 2022Improving the Factual Correctness of Radiology Report Generation with Semantic Rewardscited by 82
  12. 2020BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patientscited by 81
6 more
  1. 2024A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretationcited by 79
  2. 2021Knowledge Matters: Radiology Report Generation with General and Specific Knowledgecited by 68
  3. 2022Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priorscited by 48
  4. 2023Multimodal Image-Text Matching Improves Retrieval-based Chest X-Ray Report Generationcited by 37
  5. 2021Automated Generation of Accurate & Fluent Medical X-ray Reportscited by 24
  6. 2020Improving Broad-Coverage Medical Entity Linking with Semantic Type Prediction and Large-Scale Datasetscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 47; 2026: 62.

124 papers.

Sort:

By area:

  1. 2025Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understandingcited by 53
  2. 2024VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledgecited by 50
  3. 2025SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Paperscited by 42
  4. 2025Enhanced Contrastive Learning with Multi-view Longitudinal Data for Chest X-ray Report Generationcited by 39
  5. 2024ReXrank: A Public Leaderboard for AI-Powered Radiology Report Generationcited by 24
  6. 2025RadVLM: A Multitask Conversational Vision-Language Model for Radiologycited by 24
  7. 2025RadEval: A framework for radiology text evaluationcited by 21
  8. 2024Overview of the First Shared Task on Clinical Text Generation: RRG24 and "Discharge Me!"cited by 19
  9. 2025Boosting Vision Semantic Density with Anatomy Normality Modeling for Medical Vision-language Pre-trainingcited by 14
  10. 2025RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignmentcited by 12
  11. 2024R2GenCSR: Mining Contextual and Residual Information for LLMs-based Radiology Report Generationcited by 11
  12. 2024FMBench: Benchmarking Fairness in Multimodal Large Language Models on Medical Taskscited by 11
38 more
  1. 2025Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generationcited by 11
  2. 2025PriorRG: Prior-Guided Contrastive Pre-training and Coarse-to-Fine Decoding for Chest X-ray Report Generationcited by 9
  3. 2024Argus: Benchmarking and Enhancing Vision-Language Models for 3D Radiology Report Generationcited by 8
  4. 2024The Impact of Auxiliary Patient Data on Automated Chest X-Ray Report Generation and How to Incorporate Itcited by 8
  5. 2025Lunguage: A Benchmark for Structured and Sequential Chest X-ray Interpretationcited by 8
  6. 2025RadAgents: Multimodal Agentic Reasoning for Chest X-ray Interpretation with Radiologist-like Workflowscited by 8
  7. 2026CRIMSON: A Clinically-Grounded LLM-Based Metric for Generative Radiology Report Evaluationcited by 8
  8. 2026Can Jev Judge Radiology Reports? Evaluating a System One Model for Clinical Factualitycited by 8
  9. 2024Uncovering Knowledge Gaps in Radiology Report Generation Models through Knowledge Graphscited by 7
  10. 2025DDaTR: Dynamic Difference-aware Temporal Residual Network for Longitudinal Radiology Report Generationcited by 7
  11. 2025CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretationcited by 7
  12. 2024A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Datacited by 6
  13. 2025CRG Score: A Distribution-Aware Clinical Metric for Radiology Report Generationcited by 6
  14. 2025Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learningcited by 6
  15. 2025Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generationcited by 6
  16. 2026RadAgent: A tool-using AI agent for stepwise interpretation of chest computed tomographycited by 6
  17. 2024Best Practices for Large Language Models in Radiologycited by 5
  18. 2025From large language models to multimodal AI: A scoping review on the potential of generative AI in medicinecited by 5
  19. 2025UniVRSE: Unified Vision-conditioned Response Semantic Entropy for Hallucination Detection in Medical Vision-Language Modelscited by 5
  20. 2025CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographscited by 5
  21. 2025CLEAR: A Clinically-Grounded Tabular Framework for Radiology Report Evaluationcited by 5
  22. 2025S-RRG-Bench: Structured Radiology Report Generation with Fine-Grained Evaluation Frameworkcited by 5
  23. 2025Mammo-FM: Breast-specific foundational model for Integrated Mammographic Diagnosis, Prognosis, and Reportingcited by 5
  24. 2025Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generationcited by 5
  25. 2026MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Datacited by 5
  26. 2024EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledgecited by 4
  27. 2024Semantic Consistency-Based Uncertainty Quantification for Factuality in Radiology Report Generationcited by 4
  28. 2025CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decodingcited by 4
  29. 2025Enhancing Radiology Report Generation and Visual Grounding using Reinforcement Learningcited by 4
  30. 2025MRG-R1: Reinforcement Learning for Clinically Aligned Medical Report Generationcited by 4
  31. 2026Jolia: Concept-Level Vision-Language Alignment for 3D CT Contrastive Learningcited by 4
  32. 2024CheXalign: Preference fine-tuning in chest X-ray interpretation models without human feedbackcited by 3
  33. 2025Reason Like a Radiologist: Chain-of-Thought and Reinforcement Learning for Verifiable Report Generationcited by 3
  34. 2025Exploring the Design Space of 3D MLLMs for CT Report Generationcited by 3
  35. 2025MedVAL: Toward Expert-Level Medical Text Validation with Language Modelscited by 3
  36. 2025PETAR: Localized Findings Generation with Mask-Aware Vision-Language Modeling for PET Automated Reportingcited by 3
  37. 2026Enhancing Fine-Grained Spatial Grounding in 3D CT Report Generation via Discriminative Guidancecited by 3
  38. 2026Enhancing Reinforcement Learning for Radiology Report Generation with Evidence-aware Rewards and Self-correcting Preference Learningcited by 3

50 most cited of 124 All 124 (JSON)