mlsanityData to 14 Oct 2026

Evaluating Vision Language Model Adaptations for Radiology Report Generation in Low-Resource Languages

arXiv 2505.01096 ↗First posted 2 May 2025cs.CV cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

The integration of artificial intelligence in healthcare has opened new horizons for improving medical diagnostics and patient care. However, challenges persist in developing systems capable of generating accurate and contextually relevant radiology reports, particularly in low-resource languages. In this study, we present a comprehensive benchmark to evaluate the performance of instruction-tuned Vision-Language Models (VLMs) in the specialized task of radiology report generation across three low-resource languages: Italian, German, and Spanish. Employing the LLaVA architectural framework, we conducted a systematic evaluation of pre-trained models utilizing general datasets, domain-specific datasets, and low-resource language-specific datasets. In light of the unavailability of models that possess prior knowledge of both the medical domain and low-resource languages, we analyzed various adaptations to determine the most effective approach for these contexts. The results revealed that language-specific models substantially outperformed both general and domain-specific models in generating radiology reports, emphasizing the critical role of linguistic adaptation. Additionally, models fine-tuned with medical terminology exhibited enhanced performance across all languages compared to models with generic knowledge, highlighting the importance of domain-specific training. We also explored the influence of the temperature parameter on the coherence of report generation, providing insights for optimal model settings. Our findings highlight the importance of tailored language and domain-specific training for improving the quality and accuracy of radiological reports in multilingual settings. This research not only advances our understanding of VLMs adaptability in healthcare but also points to significant avenues for future investigations into model tuning and language-specific adaptations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 33 of its 47 references that are in the map. First posted on arXiv 2 May 2025.

Builds on

33 of 47 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024On Large Visual Language Models for Medical Imaging Analysis: An Empirical Study21 Feb
  2. 2023MAIRA-1: A specialised large multimodal model for radiology report generation22 Nov
  3. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 30

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  4. 2023Mistral 7Bcited by 3,164
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  6. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  7. 2013Generating Sequences With Recurrent Neural Networkscited by 554
  8. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  9. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  10. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
  11. 2023Segment Anything in Medical Imagescited by 227
  12. 2022Cross-modal Memory Networks for Radiology Report Generationcited by 185
18 more
  1. 2021Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generationcited by 162
  2. 2023On the Challenges and Perspectives of Foundation Models for Medical Image Analysiscited by 135
  3. 2024Exploring scalable medical image encoders beyond text supervisioncited by 132
  4. 2023Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMscited by 104
  5. 2022Improving Chest X-Ray Report Generation by Leveraging Warm Startingcited by 95
  6. 2020Low-resource Languages: A Review of Past Work and Future Challengescited by 91
  7. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  8. 2024Multilingual Instruction Tuning With Just a Pinch of Multilingualitycited by 73
  9. 2023BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languagescited by 60
  10. 2023Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpacacited by 60
  11. 2023Visual Tuningcited by 34
  12. 2024A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applicationscited by 25
  13. 2022Multimodal Explainability via Latent Shift applied to COVID-19 stratificationcited by 16
  14. 2024Multilingual Diversity Improves Vision-Language Representationscited by 14
  15. 2022Multi-objective optimization determines when, which and how to fuse deep networks: an application to predict COVID-19 outcomescited by 12
  16. 2023A Deep Learning Approach for Virtual Contrast Enhancement in Contrast Enhanced Spectral Mammographycited by 12
  17. 2024Multi-Dataset Multi-Task Learning for COVID-19 Prognosiscited by 11
  18. 2024Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learningcited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime
  2. 2024Gla-AI4BioMed at RRG24: Visual Instruction-tuned Adaptation for Radiology Report Generation
  3. 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?
  4. 2024Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology
  5. 2026RadLite: Multi-Task LoRA Fine-Tuning of Small Language Models for CPU-Deployable Radiology AI
  6. 2025Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generation
  7. 2023Consensus, dissensus and synergy between clinicians and specialist foundation models in radiology report generation
  8. 2024SERPENT-VLM : Self-Refining Radiology Report Generation Using Vision Language Models