mlsanityData to 14 Oct 2026

NeoRed: A Knowledge-Logic-Alignment Multimodal Large Language Model for Neonatal Respiratory Disease Diagnosis

arXiv 2609.03527 ↗First posted 3 Sep 2026cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Neonatal respiratory diseases are a major cause of neonatal morbidity and mortality, posing substantial challenges in clinical practice. Despite recent advances, existing Multimodal Large Language Models (MLLMs) face two key limitations in neonatal diagnosis: (1) domain gap arising from predominantly adult training data; (2) insufficient integration of multidimensional clinical context for accurate diagnosis. To address these challenges, we collect two real-world clinical datasets (NeoCXR and NeoCXR-EV) and propose NeoRed, to the best of our knowledge, the first MLLM tailored for neonatal respiratory disease, filling the gap in neonatal diagnostic reports generation. To enhance joint diagnosis from heterogeneous clinical context and chest X-rays, we design a novel Knowledge-Logic-Alignment (KLA) framework which constrains model behavior from three perspectives: 1) Knowledge Prior Injection (KPI) incorporates neonatologist-inspired diagnostic priors into multimodal representations, guiding disease-specific attention across modalities; 2) Diagnostic Logic Constraint (DLC) aligns the semantics of generated reports with multimodal diagnostic logic; and 3) Visual Semantic Alignment (VSA) establishes semantic correspondence between visual features and imaging conclusions. Extensive experiments demonstrate that NeoRed enables accurate neonatal diagnostic reports generation, achieving ROUGE-L of 53.29% and Clinical Efficacy F1 score of 65.19% on NeoCXR, outperforming existing MLLMs. NeoRed also preserves competitive report generation performance on adult benchmarks (MIMIC-CXR and IU-Xray). Datasets will be available upon application.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 37 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

17 of 37 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images6 Feb
  2. 2025EyecareGPT: Boosting Comprehensive Ophthalmology Understanding with Tailored Dataset, Benchmark and Model18 Apr
  3. 2024Towards a Multimodal Large Language Model with Pixel-Level Insight for Biomedicine12 Dec
  4. 2024LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasound19 Oct
  5. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale27 Jun
  6. 2024PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imaging5 Jan
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun
  8. 2023BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks26 May

Other references · 9

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  6. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  7. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  8. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inferencecited by 139
  9. 2025Enhanced Contrastive Learning with Multi-view Longitudinal Data for Chest X-ray Report Generationcited by 42

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Lung-R1: A Knowledge Graph-Guided LLM for Pulmonary Diagnostic Reasoning
  2. 2024RespLLM: Unifying Audio and Text with Multimodal LLMs for Generalized Respiratory Health Prediction
  3. 2025A Specialized Large Language Model for Clinical Reasoning and Diagnosis in Rare Diseases
  4. 2024A Medical Multimodal Large Language Model for Pediatric Pneumonia
  5. 2024Knowledge-Augmented Multimodal Clinical Rationale Generation for Disease Diagnosis with Small Language Models
  6. 2025Knowledge Graph Augmented Large Language Models for Disease Prediction
  7. 2026M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding
  8. 2023Large Language Models are Clinical Reasoners: Reasoning-Aware Diagnosis Framework with Prompt-Generated Rationales