mlsanityData to 7 Oct 2026

DualPrompt-MedCap: A Dual-Prompt Enhanced Approach for Medical Image Captioning

arXiv 2504.09598 ↗First posted 13 Apr 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Medical image captioning via vision-language models has shown promising potential for clinical diagnosis assistance. However, generating contextually relevant descriptions with accurate modality recognition remains challenging. We present DualPrompt-MedCap, a novel dual-prompt enhancement framework that augments Large Vision-Language Models (LVLMs) through two specialized components: (1) a modality-aware prompt derived from a semi-supervised classification model pretrained on medical question-answer pairs, and (2) a question-guided prompt leveraging biomedical language model embeddings. To address the lack of captioning ground truth, we also propose an evaluation framework that jointly considers spatial-semantic relevance and medical narrative quality. Experiments on multiple medical datasets demonstrate that DualPrompt-MedCap outperforms the baseline BLIP-3 by achieving a 22% improvement in modality recognition accuracy while generating more comprehensive and question-aligned descriptions. Our method enables the generation of clinically accurate reports that can serve as medical experts' prior knowledge and automatic annotations for downstream vision-language tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 25 references that are in the map. First posted on arXiv 13 Apr 2025.

Builds on

18 of 25 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  4. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  5. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  6. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  7. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
  8. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
  9. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  10. 2020Generating Radiology Reports via Memory-driven Transformercited by 363
  11. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
  12. 2019ScispaCy: Fast and Robust Models for Biomedical Natural Language Processingcited by 244
6 more
  1. 2017On the Automatic Generation of Medical Imaging Reportscited by 215
  2. 2018Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generationcited by 141
  3. 2018TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rayscited by 139
  4. 2019Clinically Accurate Chest X-Ray Report Generationcited by 138
  5. 2024xGen-MM (BLIP-3): A Family of Open Large Multimodal Modelscited by 116
  6. 2023Tag2Text: Guiding Vision-Language Model via Image Taggingcited by 84

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Biomed-DPT: Dual Modality Prompt Tuning for Biomedical Vision-Language Models
  2. 2026Towards Clinically Faithful Medical Image Captioning via Enhanced Vision-Language Alignment
  3. 2024Fusion of Domain-Adapted Vision and Language Models for Medical Visual Question Answering
  4. 2025MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation
  5. 2022Medical Image Understanding with Pretrained Vision Language Models: A Comprehensive Study
  6. 2025Describe Anything in Medical Images
  7. 2025Guiding Medical Vision-Language Models with Explicit Visual Prompts: Framework Design and Comprehensive Exploration of Prompt Variations
  8. 2021Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training