# ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Report Generation Based on Multi-institution and Multi-system Data

arXiv: https://arxiv.org/abs/2310.05242 · HTML: https://mlsanity.com/paper/2310.05242/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 61 references that are in the map.
- First posted on arXiv 8 Oct 2023.

## Builds on (most cited first, 30 of 33)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca](https://mlsanity.com/paper/2304.08177/) (2023)
- [On the Automatic Generation of Medical Imaging Reports](https://mlsanity.com/paper/1711.08195/) (2017)
- [Cross-modal Memory Networks for Radiology Report Generation](https://mlsanity.com/paper/2204.13258/) (2022)
- [Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation](https://mlsanity.com/paper/2106.06963/) (2021)
- [When Radiology Report Generation Meets Knowledge Graph](https://mlsanity.com/paper/2002.08277/) (2020)
- [Clinically Accurate Chest X-Ray Report Generation](https://mlsanity.com/paper/1904.02633/) (2019)
- [Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge](https://mlsanity.com/paper/1609.06647/) (2016)
- [ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Models](https://mlsanity.com/paper/2302.07257/) (2023)
- [Competence-based Multimodal Curriculum Learning for Medical Report Generation](https://mlsanity.com/paper/2206.14579/) (2022)
- [Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichment](https://mlsanity.com/paper/1907.09085/) (2019)
- [Cross-modal Prototype Driven Network for Radiology Report Generation](https://mlsanity.com/paper/2207.04818/) (2022)
- [A Review of Uncertainty Estimation and its Application in Medical Imaging](https://mlsanity.com/paper/2302.08119/) (2023)
- [An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT](https://mlsanity.com/paper/2304.08448/) (2023)
- [When Brain-inspired AI Meets AGI](https://mlsanity.com/paper/2303.15935/) (2023)
- [Radiology-GPT: A Large Language Model for Radiology](https://mlsanity.com/paper/2306.08666/) (2023)
- [Progressive Transformer-Based Generation of Radiology Reports](https://mlsanity.com/paper/2102.09777/) (2021)
- [BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models](https://mlsanity.com/paper/2306.10968/) (2023)

## Built on by (most cited first, 14 of 14)

- [Understanding LLMs: A Comprehensive Overview from Training to Inference](https://mlsanity.com/paper/2401.02038/) (2024)
- [RET-CLIP: A Retinal Image Foundation Model Pre-trained with Clinical Diagnostic Reports](https://mlsanity.com/paper/2405.14137/) (2024)
- [Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysis](https://mlsanity.com/paper/2402.11398/) (2024)
- [Holistic Evaluation of GPT-4V for Biomedical Imaging](https://mlsanity.com/paper/2312.05256/) (2023)
- [A Survey on Trustworthiness in Foundation Models for Medical Image Analysis](https://mlsanity.com/paper/2407.15851/) (2024)
- [Ophtha-LLaMA2: A Large Language Model for Ophthalmology](https://mlsanity.com/paper/2312.04906/) (2023)
- [Tree-of-Reasoning: Towards Complex Medical Diagnosis via Multi-Agent Reasoning with Evidence Tree](https://mlsanity.com/paper/2508.03038/) (2025)
- [Potential of Multimodal Large Language Models for Data Mining of Medical Images and Free-text Reports](https://mlsanity.com/paper/2407.05758/) (2024)
- [Breast Ultrasound Report Generation using LangChain](https://mlsanity.com/paper/2312.03013/) (2023)
- [Evaluating multiple large language models in pediatric ophthalmology](https://mlsanity.com/paper/2311.04368/) (2023)
- [Large Language Models for Bioinformatics](https://mlsanity.com/paper/2501.06271/) (2025)
- [Integrating ChatGPT into Secure Hospital Networks: A Case Study on Improving Radiology Report Analysis](https://mlsanity.com/paper/2402.09358/) (2024)
- [Dual-Domain CLIP-Assisted Residual Optimization Perception Model for Metal Artifact Reduction](https://mlsanity.com/paper/2408.14342/) (2024)
- [AI-Driven Radiology Report Generation for Traumatic Brain Injuries](https://mlsanity.com/paper/2510.08498/) (2025)

## Abstract (arXiv)

Radiology report generation, as a key step in medical image analysis, is critical to the quantitative analysis of clinically informed decision-making levels. However, complex and diverse radiology reports with cross-source heterogeneity pose a huge generalizability challenge to the current methods under massive data volume, mainly because the style and normativity of radiology reports are obviously distinctive among institutions, body regions inspected and radiologists. Recently, the advent of large language models (LLM) offers great potential for recognizing signs of health conditions. To resolve the above problem, we collaborate with the Second Xiangya Hospital in China and propose ChatRadio-Valuer based on the LLM, a tailored model for automatic radiology report generation that learns generalizable representations and provides a basis pattern for model adaptation in sophisticated analysts' cases. Specifically, ChatRadio-Valuer is trained based on the radiology reports from a single institution by means of supervised fine-tuning, and then adapted to disease diagnosis tasks for human multi-system evaluation (i.e., chest, abdomen, muscle-skeleton, head, and maxillofacial & neck) from six different institutions in clinical-level events. The clinical dataset utilized in this study encompasses a remarkable total of 332,673 observations. From the comprehensive results on engineering indicators, clinical efficacy and deployment cost metrics, it can be shown that ChatRadio-Valuer consistently outperforms state-of-the-art models, especially ChatGPT (GPT-3.5-Turbo) and GPT-4 et al., in terms of the diseases diagnosis from radiology reports. ChatRadio-Valuer provides an effective avenue to boost model generalization performance and alleviate the annotation workload of experts to enable the promotion of clinical AI applications in radiology reports.