mlsanityData to 14 Oct 2026

LiteGPT: Large Vision-Language Model for Joint Chest X-ray Localization and Classification Task

arXiv 2407.12064 ↗First posted 16 Jul 2024eess.IV cs.CL cs.CVcited by 4

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision-language models have been extensively explored across a wide range of tasks, achieving satisfactory performance; however, their application in medical imaging remains underexplored. In this work, we propose a unified framework - LiteGPT - for the medical imaging. We leverage multiple pre-trained visual encoders to enrich information and enhance the performance of vision-language models. To the best of our knowledge, this is the first study to utilize vision-language models for the novel task of joint localization and classification in medical images. Besides, we are pioneers in providing baselines for disease localization in chest X-rays. Finally, we set new state-of-the-art performance in the image classification task on the well-benchmarked VinDr-CXR dataset. All code and models are publicly available online: https://github.com/leduckhai/LiteGPT

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 71 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

19 of 71 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024DeViDe: Faceted medical knowledge for improved medical vision-language pre-training4 Apr
  2. 2024PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imaging5 Jan
  3. 2023XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models13 Jun

Other references · 16

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  5. 2023Qwen Technical Reportcited by 3,344
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  8. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  9. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  10. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  11. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  12. 2023Textbooks Are All You Needcited by 569
4 more
  1. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
  2. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  3. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  4. 2024DWARF: Disease-weighted network for attention map refinementcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2025SilVar-Med: A Speech-Driven Visual Language Model for Explainable Abnormality Detection in Medical Imagingcited by 5
  2. 2024OrthoDoc: Multimodal Large Language Model for Assisting Diagnosis in Computed Tomographycited by 3
  3. 2025Point, Detect, Count: Multi-Task Medical Image Understanding with Instruction-Tuned Vision-Language Modelscited by 2
  4. 2025Benchmark Success, Clinical Failure: When Reinforcement Learning Optimizes for Benchmarks, Not Patientscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders
  2. 2025XBench: A Comprehensive Benchmark for Visual-Language Explanations in Chest Radiography
  3. 2023MedXChat: A Unified Multimodal Large Language Model Framework towards CXRs Understanding and Generation
  4. 2026RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecture
  5. 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?
  6. 2023G2D: From Global to Dense Radiography Representation Learning via Vision-Language Pre-training
  7. 2026GLINT: Sparsely Gated Vision-Language Alignment for Fine-Grained Radiology Representations
  8. 2023ELVIS: Empowering Locality of Vision Language Pre-training with Intra-modal Similarity