mlsanityData to 7 Oct 2026

Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation

arXiv 2010.10042 ↗First posted 20 Oct 2020cs.CLcited by 139

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Neural image-to-text radiology report generation systems offer the potential to improve radiology reporting by reducing the repetitive process of report drafting and identifying possible medical errors. However, existing report generation systems, despite achieving high performances on natural language generation metrics such as CIDEr or BLEU, still suffer from incomplete and inconsistent generations. Here we introduce two new simple rewards to encourage the generation of factually complete and consistent radiology reports: one that encourages the system to generate radiology domain entities consistent with the reference, and one that uses natural language inference to encourage these entities to be described in inferentially consistent ways. We combine these with the novel use of an existing semantic equivalence metric (BERTScore). We further propose a report generation system that optimizes these rewards via reinforcement learning. On two open radiology report datasets, our system substantially improved the F1 score of a clinical information extraction performance by +22.1 (Delta +63.9%). We further show via a human evaluation and a qualitative analysis that our system leads to generations that are more factually complete and consistent compared to the baselines.

Key facts. Cited by 139 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 17; 2023: 39; 2024: 31; 2025: 25; 2026: 25.

139 papers.

Sort:

By area:

  1. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 336
  2. 2023Towards Generalist Biomedical AIcited by 286
  3. 2022Transformers in Medical Imaging: A Surveycited by 245
  4. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
  5. 2023Conformal Language Modelingcited by 163
  6. 2023MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiologycited by 149
  7. 2023Interactive and Explainable Region-guided Radiology Report Generationcited by 139
  8. 2024Exploring scalable medical image encoders beyond text supervisioncited by 129
  9. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  10. 2023When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applicationscited by 110
  11. 2022RoentGen: Vision-Language Foundation Model for Chest X-ray Generationcited by 89
  12. 2024Advancing Multimodal Medical Capabilities of Geminicited by 88
38 more
  1. 2022Improving the Factual Correctness of Radiology Report Generation with Semantic Rewardscited by 82
  2. 2022Competence-based Multimodal Curriculum Learning for Medical Report Generationcited by 74
  3. 2023MAIRA-1: A specialised large multimodal model for radiology report generationcited by 72
  4. 2021Contrastive Attention for Automatic Chest X-ray Report Generationcited by 71
  5. 2022Cross-modal Prototype Driven Network for Radiology Report Generationcited by 58
  6. 2023Towards a Visual-Language Foundation Model for Computational Pathologycited by 52
  7. 2022Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priorscited by 48
  8. 2024MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imagingcited by 45
  9. 2024Automated Radiology Report Generation: A Review of Recent Advancescited by 43
  10. 2022Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inferencecited by 40
  11. 2023CLIP in Medical Imaging: A Surveycited by 40
  12. 2023ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoningcited by 39
  13. 2023Multimodal Image-Text Matching Improves Retrieval-based Chest X-Ray Report Generationcited by 37
  14. 2021Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Reviewcited by 35
  15. 2023Towards Unifying Medical Vision-and-Language Pre-training via Soft Promptscited by 35
  16. 2024WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answeringcited by 34
  17. 2024CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Datasetcited by 33
  18. 2024HERGen: Elevating Radiology Report Generation with Longitudinal Datacited by 32
  19. 2023Exploring the Boundaries of GPT-4 in Radiologycited by 31
  20. 2023WsiCaption: Multiple Instance Generation of Pathology Reports for Gigapixel Whole-Slide Imagescited by 31
  21. 2024Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generationcited by 30
  22. 2024Libra: Leveraging Temporal Images for Biomedical Radiology Analysiscited by 29
  23. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  24. 2023Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT modelscited by 27
  25. 2023RadAdapt: Radiology Report Summarization via Lightweight Domain Adaptation of Large Language Modelscited by 24
  26. 2023Longitudinal Data and a Semantic Similarity Reward for Chest X-Ray Report Generationcited by 24
  27. 2023Medical Vision Language Pretraining: A surveycited by 24
  28. 2023Controllable Chest X-Ray Report Generation from Longitudinal Representationscited by 23
  29. 2024Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generationcited by 23
  30. 2024An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generationcited by 23
  31. 2023RECAP: Towards Precise Radiology Report Generation via Dynamic Disease Progression Reasoningcited by 22
  32. 2025RadEval: A framework for radiology text evaluationcited by 21
  33. 2025MedVAE: Efficient Automated Interpretation of Medical Images with Large-Scale Generalizable Autoencoderscited by 20
  34. 2024ChEX: Interactive Localization and Region Description in Chest X-rayscited by 18
  35. 2022Attributed Abnormality Graph Embedding for Clinically Accurate X-Ray Report Generationcited by 17
  36. 2023DocLens: Multi-aspect Fine-grained Evaluation for Medical Text Generationcited by 16
  37. 2022A survey on attention mechanisms for medical applications: are we moving towards better algorithms?cited by 15
  38. 2022Toward expanding the scope of radiology report summarization to multiple anatomies and modalitiescited by 14

50 most cited of 139 All 139 (JSON)