mlsanityData to 7 Oct 2026

Democratizing MLLMs in Healthcare: TinyLLaVA-Med for Efficient Healthcare Diagnostics in Resource-Constrained Settings

arXiv 2409.12184 ↗First posted 2 Sep 2024cs.LG cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Deploying Multi-Modal Large Language Models (MLLMs) in healthcare is hindered by their high computational demands and significant memory requirements, which are particularly challenging for resource-constrained devices like the Nvidia Jetson Xavier. This problem is particularly evident in remote medical settings where advanced diagnostics are needed but resources are limited. In this paper, we introduce an optimization method for the general-purpose MLLM, TinyLLaVA, which we have adapted and renamed TinyLLaVA-Med. This adaptation involves instruction-tuning and fine-tuning TinyLLaVA on a medical dataset by drawing inspiration from the LLaVA-Med training pipeline. Our approach successfully minimizes computational complexity and power consumption, with TinyLLaVA-Med operating at 18.9W and using 11.9GB of memory, while achieving accuracies of 64.54% on VQA-RAD and 70.70% on SLAKE for closed-ended questions. Therefore, TinyLLaVA-Med achieves deployment viability in hardware-constrained environments with low computational resources, maintaining essential functionalities and delivering accuracies close to state-of-the-art models.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 8 of its 17 references that are in the map. First posted on arXiv 2 Sep 2024.

Builds on

8 of 17 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models16 Apr
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 6

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  3. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  4. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  5. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  6. 2024TinyLLaVA: A Framework of Small-scale Large Multimodal Modelscited by 164

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Pulmonary Tuberculosis Edge Diagnosis System Based on MindSpore Framework: Low-cost and High-precision Implementation with Ascend 310 Chipcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Multimodal Large Language Models for Medicine: A Comprehensive Survey
  2. 2024A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models
  3. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine
  4. 2025Medicine on the Edge: Comparative Performance Analysis of On-Device LLMs for Clinical Reasoning
  5. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
  6. 2025Benchmarking and Adapting On-Device LLMs for Clinical Decision Support
  7. 2024MedAide: Leveraging Large Language Models for On-Premise Medical Assistance on Edge Devices
  8. 2024MedMobile: A mobile-sized language model with clinical capabilities