Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer's Disease
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
The rapid advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) have shown great potential in medical diagnostics, particularly in radiology, where datasets such as X-rays are paired with human-generated diagnostic reports. However, a significant research gap exists in the neuroimaging field, especially for conditions such as Alzheimer's disease, due to the lack of comprehensive diagnostic reports that can be utilized for model fine-tuning. This paper addresses this gap by generating synthetic diagnostic reports using GPT-4o-mini on structured data from the OASIS-4 dataset, which comprises 663 patients. Using the synthetic reports as ground truth for training and validation, we then generated neurological reports directly from the images in the dataset leveraging the pre-trained BiomedCLIP and T5 models. Our proposed method achieved a BLEU-4 score of 0.1827, ROUGE-L score of 0.3719, and METEOR score of 0.4163, revealing its potential in generating clinically relevant and accurate diagnostic reports.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 40 references that are in the map. First posted on arXiv 12 Nov 2024.
Builds on
15 of 40 references.
Closest ancestors · 3
Same direction, newest first
- 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review4 Mar
- 2024On Large Visual Language Models for Medical Imaging Analysis: An Empirical Study21 Feb
- 2023Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V29 Oct
Other references · 12
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
- 2019Publicly Available Clinical BERT Embeddingscited by 1,002
- 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
- 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
- 2023Towards Generalist Biomedical AIcited by 289
- 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 230
- 2019An Attentive Survey of Attention Modelscited by 124
- 2023ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Modelscited by 116
- 2023Large language models improve Alzheimer's disease diagnosis using multi-modality datacited by 9
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Data-Efficient Fine-Tuning of Vision-Language Models for Diagnosis of Alzheimer's Disease
- 2026NeuroVLM-Bench: Evaluation of Vision-Enabled Large Language Models for Clinical Reasoning in Neurological Disorders
- 2026EMAD: Evidence-Centric Grounded Multimodal Diagnosis for Alzheimer's Disease
- 2025An Explainable Diagnostic Framework for Neurodegenerative Dementias via Reinforcement-Optimized LLM Reasoning
- 2026Chain-of-Thought Reasoning with Large Language Models for Clinical Alzheimer's Disease Assessment and Diagnosis
- 2025AD-GPT: Large Language Models in Alzheimer's Disease
- 2025REMEMBER: Retrieval-based Explainable Multimodal Evidence-guided Modeling for Brain Evaluation and Reasoning in Zero- and Few-shot Neurodegenerative Diagnosis
- 2026AD-CARE: A Guideline-grounded, Modality-agnostic LLM Agent for Real-world Alzheimer's Disease Diagnosis with Multi-cohort Assessment, Fairness Analysis, and Reader Study