mlsanityData to 14 Oct 2026

MedCutMix: A Data-Centric Approach to Improve Radiology Vision-Language Pre-training with Disease Awareness

arXiv 2509.16673 ↗First posted 20 Sep 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision-Language Pre-training (VLP) is drawing increasing interest for its ability to minimize manual annotation requirements while enhancing semantic understanding in downstream tasks. However, its reliance on image-text datasets poses challenges due to privacy concerns and the high cost of obtaining paired annotations. Data augmentation emerges as a viable strategy to address this issue, yet existing methods often fall short of capturing the subtle and complex variations in medical data due to limited diversity. To this end, we propose MedCutMix, a novel multi-modal disease-centric data augmentation method. MedCutMix performs diagnostic sentence CutMix within medical reports and establishes the cross-attention between the diagnostic sentence and medical image to guide attentive manifold mix within the imaging modality. Our approach surpasses previous methods across four downstream radiology diagnosis datasets, highlighting its effectiveness in enhancing performance and generalizability in radiology VLP.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 23 references that are in the map. First posted on arXiv 20 Sep 2025.

Builds on

18 of 23 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding24 Jan

Other references · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  5. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  6. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  7. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  8. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  9. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
  10. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
  11. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
  12. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
5 more
  1. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 309
  2. 2022Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learningcited by 159
  3. 2021Joint Learning of Localized Representations from Medical Images and Reportscited by 81
  4. 2023Advancing Radiograph Representation Learning with Masked Record Modelingcited by 75
  5. 2023Towards Unifying Medical Vision-and-Language Pre-training via Soft Promptscited by 36

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?
  2. 2024DeViDe: Faceted medical knowledge for improved medical vision-language pre-training
  3. 2023Multi-modal Pre-training for Medical Vision-language Understanding and Generation: An Empirical Study with A New Benchmark
  4. 2023Unified Medical Image Pre-training in Language-Guided Common Semantic Space
  5. 2026SHOVIR: A Benchmark for Evaluating Vision Shortcut Learning in Radiology Report Generation
  6. 2023Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime
  7. 2025Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generation
  8. 2024Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology