mlsanityData to 14 Oct 2026

A Medical Multimodal Large Language Model for Pediatric Pneumonia

arXiv 2409.02608 ↗First posted 4 Sep 2024cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Pediatric pneumonia is the leading cause of death among children under five years worldwide, imposing a substantial burden on affected families. Currently, there are three significant hurdles in diagnosing and treating pediatric pneumonia. Firstly, pediatric pneumonia shares similar symptoms with other respiratory diseases, making rapid and accurate differential diagnosis challenging. Secondly, primary hospitals often lack sufficient medical resources and experienced doctors. Lastly, providing personalized diagnostic reports and treatment recommendations is labor-intensive and time-consuming. To tackle these challenges, we proposed a Medical Multimodal Large Language Model for Pediatric Pneumonia (P2Med-MLLM). It was capable of handling diverse clinical tasks, such as generating free-text radiology reports and medical records within a unified framework. Specifically, P2Med-MLLM can process both pure text and image-text data, trained on an extensive and large-scale dataset (P2Med-MD), including real clinical information from 163,999 outpatient and 8,684 inpatient cases. This dataset comprised 2D chest X-ray images, 3D chest CT images, corresponding radiology reports, and outpatient and inpatient records. We designed a three-stage training strategy to enable P2Med-MLLM to comprehend medical knowledge and follow instructions for various clinical tasks. To rigorously evaluate P2Med-MLLM's performance, we developed P2Med-MBench, a benchmark consisting of 642 meticulously verified samples by pediatric pulmonology specialists, covering six clinical decision-support tasks and a balanced variety of diseases. The automated scoring results demonstrated the superiority of P2Med-MLLM. This work plays a crucial role in assisting primary care doctors with prompt disease diagnosis and treatment planning, reducing severe symptom mortality rates, and optimizing the allocation of medical resources.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 4 Sep 2024.

Builds on

41 of 58 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  10. 2023Mistral 7Bcited by 3,164
  11. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  12. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
29 more
  1. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  2. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  3. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  4. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  5. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  6. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  7. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
  8. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 565
  9. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  10. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
  11. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  12. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  13. 2023Towards Generalist Biomedical AIcited by 289
  14. 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 179
  15. 2023SantaCoder: don't reach for the stars!cited by 170
  16. 2022Scaling Laws and Interpretability of Learning from Repeated Datacited by 152
  17. 2024MAIRA-2: Grounded Radiology Report Generationcited by 145
  18. 2024Towards Conversational Diagnostic AIcited by 79
  19. 2023Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosiscited by 77
  20. 2023Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcarecited by 62
  21. 2023LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generationcited by 57
  22. 2023ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoderscited by 50
  23. 2023A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnosticscited by 43
  24. 2023ChatCAD+: Towards a Universal and Reliable Interactive CAD using LLMscited by 35
  25. 2023OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialoguecited by 34
  26. 2023SkinGPT-4: An Interactive Dermatology Diagnostic System with Visual Large Language Modelcited by 18
  27. 2024MedTsLLM: Leveraging LLMs for Multimodal Medical Time Series Analysiscited by 15
  28. 2023Path to Medical AGI: Unify Domain-specific Medical LLMs with the Lowest Costcited by 12
  29. 2021Identification of Pediatric Respiratory Diseases Using Fine-grained Diagnosis Systemcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025FineMedLM-o1: Enhancing Medical Knowledge Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Trainingcited by 19

Similar, not cited

Close in wording, no citation either way

  1. 2024Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement
  2. 2024From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practice
  3. 2026NeoRed: A Knowledge-Logic-Alignment Multimodal Large Language Model for Neonatal Respiratory Disease Diagnosis
  4. 2024Utility of Multimodal Large Language Models in Analyzing Chest X-ray with Incomplete Contextual Information
  5. 2025Multimodal Large Language Models for Medical Report Generation via Customized Prompt Tuning
  6. 2024Med-2E3: A 2D-Enhanced 3D Medical Multimodal Large Language Model
  7. 2026MEDSYN: Benchmarking Multi-EviDence SYNthesis in Complex Clinical Cases for Multimodal Large Language Models
  8. 2024RespLLM: Unifying Audio and Text with Multimodal LLMs for Generalized Respiratory Health Prediction