mlsanityData to 7 Oct 2026

RaTEScore: A Metric for Radiology Report Generation

arXiv 2406.16845 ↗First posted 24 Jun 2024cs.CLcited by 65

Not on the map. Why?

Abstract (arXiv)

This paper introduces a novel, entity-aware metric, termed as Radiological Report (Text) Evaluation (RaTEScore), to assess the quality of medical reports generated by AI models. RaTEScore emphasizes crucial medical entities such as diagnostic outcomes and anatomical details, and is robust against complex medical synonyms and sensitive to negation expressions. Technically, we developed a comprehensive medical NER dataset, RaTE-NER, and trained an NER model specifically for this purpose. This model enables the decomposition of complex radiological reports into constituent medical entities. The metric itself is derived by comparing the similarity of entity embeddings, obtained from a language model, based on their types and relevance to clinical significance. Our evaluations demonstrate that RaTEScore aligns more closely with human preference than existing metrics, validated both on established public benchmarks and our newly proposed RaTE-Eval benchmark.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 22 of its 53 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

22 of 53 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  5. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  6. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  7. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  8. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 336
  9. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 329
  10. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
  11. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
  12. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
10 more
  1. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  2. 2023ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generationcited by 116
  3. 2024Towards Building Multilingual Language Model for Medicinecited by 91
  4. 2023Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosiscited by 77
  5. 2023Large-Vocabulary Segmentation for Medical Images with Text Promptscited by 60
  6. 2021Launching into clinical space with medspaCy: a new clinical text processing toolkit in Pythoncited by 40
  7. 2023Large-scale Long-tailed Disease Diagnosis on Radiology Imagescited by 17
  8. 2023How far is Language Model from 100% Few-shot Named Entity Recognition in Medical Domaincited by 14
  9. 2024LLM-RadJudge: Achieving Radiologist-Level Evaluation for X-Ray Report Generationcited by 14
  10. 2024Knowledge-enhanced Visual-Language Pretraining for Computational Pathologycited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 23; 2026: 38.

65 papers.

Sort:

By area:

  1. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 217
  2. 2025SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Paperscited by 42
  3. 2024ReXrank: A Public Leaderboard for AI-Powered Radiology Report Generationcited by 24
  4. 2024AutoRG-Brain: Grounded Report Generation for Brain MRIcited by 23
  5. 2025Lunguage: A Benchmark for Structured and Sequential Chest X-ray Interpretationcited by 8
  6. 2025Citrus-V: Advancing Medical Foundation Models with Unified Medical Image Grounding for Clinical Reasoningcited by 8
  7. 2025RadAgents: Multimodal Agentic Reasoning for Chest X-ray Interpretation with Radiologist-like Workflowscited by 8
  8. 2026CRIMSON: A Clinically-Grounded LLM-Based Metric for Generative Radiology Report Evaluationcited by 8
  9. 2026Can Jev Judge Radiology Reports? Evaluating a System One Model for Clinical Factualitycited by 8
  10. 2024Uncovering Knowledge Gaps in Radiology Report Generation Models through Knowledge Graphscited by 7
  11. 2025CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretationcited by 7
  12. 2026Scaling medical imaging report generation with multimodal reinforcement learningcited by 7
38 more
  1. 2024Can Medical Vision-Language Pre-training Succeed with Purely Synthetic Data?cited by 6
  2. 2025Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decodingcited by 6
  3. 2025XGeM: A Multi-Prompt Foundation Model for Multimodal Medical Data Generationcited by 5
  4. 2025From large language models to multimodal AI: A scoping review on the potential of generative AI in medicinecited by 5
  5. 2025CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographscited by 5
  6. 2025S-RRG-Bench: Structured Radiology Report Generation with Fine-Grained Evaluation Frameworkcited by 5
  7. 2025Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generationcited by 5
  8. 2025RadIR: A Scalable Framework for Multi-Grained Medical Image Retrieval via Radiology Report Miningcited by 4
  9. 2026A Reasoning-Enabled Vision-Language Foundation Model for Chest X-ray Interpretationcited by 4
  10. 2025Exploring the Design Space of 3D MLLMs for CT Report Generationcited by 3
  11. 2025PETAR: Localized Findings Generation with Mask-Aware Vision-Language Modeling for PET Automated Reportingcited by 3
  12. 2026TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysiscited by 3
  13. 2026ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusioncited by 3
  14. 2026RadSEM: A Finding-by-Finding Metric for Clinical Consistency in Radiology Reportscited by 3
  15. 2025Rethinking Whole-Body CT Image Interpretation: An Abnormality-Centric Approachcited by 2
  16. 2025Recurrent Visual Feature Extraction and Stereo Attentions for CT Report Generationcited by 2
  17. 2025Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generationcited by 2
  18. 2025CX-Mind: A Pioneering Multimodal Large Language Model for Interleaved Reasoning in Chest X-ray via Curriculum-Guided Reinforcement Learningcited by 2
  19. 2026CTest-Metric: A Unified Framework to Assess Clinical Validity of Metrics for CT Report Generationcited by 2
  20. 2026A data- and compute-efficient chest X-ray foundation model beyond aggressive scalingcited by 2
  21. 2026VERT: Reliable LLM Judges for Radiology Report Evaluationcited by 2
  22. 2026Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysiscited by 2
  23. 2025MicarVLMoE: A Modern Gated Cross-Aligned Vision-Language Mixture of Experts Model for Medical Image Captioning and Report Generationcited by 1
  24. 2025HSENet: Hybrid Spatial Encoding Network for 3D Medical Vision-Language Understandingcited by 1
  25. 2026CURE: Curriculum-guided Multi-task Training for Reliable Anatomy Grounded Report Generationcited by 1
  26. 2026BTReport: A Framework for Brain Tumor Radiology Report Generation with Clinically Relevant Featurescited by 1
  27. 2026Sparse Spectral LoRA: Routed Experts for Medical VLMscited by 1
  28. 2026SliceWorld: A Predictive and Controllable World-State Model for CT Report Generationcited by 1
  29. 2026A Vision-language Framework for Comparative Reasoning in Radiologycited by 1
  30. 2026ReportQA: QA-Based Radiology Report Evaluationcited by 1
  31. 2026Seeing What Matters: Lesion-Aware High-Resolution Patch Discovery and Fusion for Chest X-ray Report Generationcited by 1
  32. 2026RadSight: Towards Perceptually Reliable Multimodal Radiology Image Understandingcited by 1
  33. 2026RadMatch: Auditable Radiology Report Evaluation via Finding-Level Matchingcited by 1
  34. 2025Privacy-Preserving Chest X-ray Report Generation via Multimodal Federated Learning with ViT and GPT-2cited by 0
  35. 2025RadEyeVideo: Enhancing general-domain Large Vision Language Model for chest X-ray analysis with video representations of eye gazecited by 0
  36. 2026CSEval: A Framework for Evaluating Clinical Semantics in Text-to-Image Generationcited by 0
  37. 2026MedGPT-oss: Training a General-Purpose Vision-Language Model for Biomedicinecited by 0
  38. 2026MedCollab: IBIS-Guided Multi-Agent Collaboration with Hierarchical Disease Relation Chains for Clinical Diagnosiscited by 0

50 most cited of 65 All 65 (JSON)