mlsanityData to 7 Oct 2026

MiniGPT-Med: Large Language Model as a General Interface for Radiology Diagnosis

arXiv 2407.04106 ↗First posted 4 Jul 2024cs.AI cs.CL cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

Recent advancements in artificial intelligence (AI) have precipitated significant breakthroughs in healthcare, particularly in refining diagnostic procedures. However, previous studies have often been constrained to limited functionalities. This study introduces MiniGPT-Med, a vision-language model derived from large-scale language models and tailored for medical applications. MiniGPT-Med demonstrates remarkable versatility across various imaging modalities, including X-rays, CT scans, and MRIs, enhancing its utility. The model is capable of performing tasks such as medical report generation, visual question answering (VQA), and disease identification within medical imagery. Its integrated processing of both image and textual clinical data markedly improves diagnostic accuracy. Our empirical assessments confirm MiniGPT-Med's superior performance in disease grounding, medical report generation, and VQA benchmarks, representing a significant step towards reducing the gap in assisting radiology practice. Furthermore, it achieves state-of-the-art performance on medical report generation, higher than the previous best model by 19% accuracy. MiniGPT-Med promises to become a general interface for radiology diagnoses, enhancing diagnostic efficiency across a wide range of medical imaging applications.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 16 of its 32 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

16 of 32 references.

References · 16

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  6. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  8. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  9. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  10. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  11. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  12. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
4 more
  1. 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 176
  2. 2024A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretationcited by 79
  3. 2021BERTHop: An Effective Vision-and-Language Model for Chest X-ray Disease Diagnosiscited by 20
  4. 2023Visual Question Answering in the Medical Domaincited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 16; 2026: 4.

28 papers.

Sort:

By area:

  1. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 84
  2. 2024GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosiscited by 28
  3. 2024Large Language Model with Region-guided Referring and Grounding for CT Report Generationcited by 27
  4. 2024Interpretable Bilingual Multimodal Large Language Model for Diverse Biomedical Taskscited by 24
  5. 2024BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalitiescited by 20
  6. 2024E3D-GPT: Enhanced 3D Visual Foundation for Medical Vision-Language Modelcited by 19
  7. 2025Multimodal Large Language Models for Medicine: A Comprehensive Surveycited by 16
  8. 2025MM-Skin: Enhancing Dermatology Vision-Language Model with an Image-Text Dataset Derived from Textbookscited by 14
  9. 2025Anyprefer: An Agentic Framework for Preference Data Synthesiscited by 10
  10. 2025MMXU: A Multi-Modal and Multi-X-ray Understanding Dataset for Disease Progressioncited by 9
  11. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Reviewcited by 9
  12. 2025U2-BENCH: Benchmarking Large Vision-Language Models on Ultrasound Understandingcited by 8
16 more
  1. 2025Patient-Level Anatomy Meets Scanning-Level Physics: Personalized Federated Low-Dose CT Denoising Empowered by Large Language Modelcited by 6
  2. 2025STING-BEE: Towards Vision-Language Model for Real-World X-ray Baggage Security Inspectioncited by 5
  3. 2025SilVar-Med: A Speech-Driven Visual Language Model for Explainable Abnormality Detection in Medical Imagingcited by 5
  4. 2024OrthoDoc: Multimodal Large Language Model for Assisting Diagnosis in Computed Tomographycited by 3
  5. 2025MCA-RG: Enhancing LLMs with Medical Concept Alignment for Radiology Report Generationcited by 3
  6. 2026Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMscited by 3
  7. 2026Enhancing Reinforcement Learning for Radiology Report Generation with Evidence-aware Rewards and Self-correcting Preference Learningcited by 3
  8. 2025Online Iterative Self-Alignment for Radiology Report Generationcited by 2
  9. 2026V-Loop: Visual Logical Loop Verification for Hallucination Detection in Medical Visual Question Answeringcited by 2
  10. 2024MiniGPT-Pancreas: Multimodal Large Language Model for Pancreas Cancer Classification and Detectioncited by 1
  11. 2025Enhancing Multi-task Learning Capability of Medical Generalist Foundation Model via Image-centric Multi-annotation Datacited by 1
  12. 2025ChestGPT: Integrating Large Language Models and Vision Transformers for Disease Detection and Localization in Chest X-Rayscited by 1
  13. 2025CogBench: A Large Language Model Benchmark for Multilingual Speech-Based Cognitive Impairment Assessmentcited by 1
  14. 2025Knowledge to Sight: Reasoning over Visual Attributes via Knowledge Decomposition for Abnormality Groundingcited by 1
  15. 2026Beyond a Single Frame: Multi-Frame Spatially Grounded Reasoning Across Volumetric MRIcited by 1
  16. 2025Interpreting Chest X-rays Like a Radiologist: A Benchmark with Clinical Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks
  2. 2023Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task
  3. 2023Artificial General Intelligence for Medical Imaging Analysis
  4. 2023XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models
  5. 20243D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Models
  6. 2024Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology
  7. 2025Evaluating Vision Language Model Adaptations for Radiology Report Generation in Low-Resource Languages
  8. 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?