mlsanityData to 14 Oct 2026

CrossMed: A Multimodal Cross-Task Benchmark for Compositional Generalization in Medical Imaging

arXiv 2511.11034 ↗First posted 14 Nov 2025cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Recent advances in multimodal large language models have enabled unified processing of visual and textual inputs, offering promising applications in general-purpose medical AI. However, their ability to generalize compositionally across unseen combinations of imaging modality, anatomy, and task type remains underexplored. We introduce CrossMed, a benchmark designed to evaluate compositional generalization (CG) in medical multimodal LLMs using a structured Modality-Anatomy-Task (MAT) schema. CrossMed reformulates four public datasets, CheXpert (X-ray classification), SIIM-ACR (X-ray segmentation), BraTS 2020 (MRI classification and segmentation), and MosMedData (CT classification) into a unified visual question answering (VQA) format, resulting in 20,200 multiple-choice QA instances. We evaluate two open-source multimodal LLMs, LLaVA-Vicuna-7B and Qwen2-VL-7B, on both Related and Unrelated MAT splits, as well as a zero-overlap setting where test triplets share no Modality, Anatomy, or Task with the training data. Models trained on Related splits achieve 83.2 percent classification accuracy and 0.75 segmentation cIoU, while performance drops significantly under Unrelated and zero-overlap conditions, demonstrating the benchmark difficulty. We also show cross-task transfer, where segmentation performance improves by 7 percent cIoU even when trained using classification-only data. Traditional models (ResNet-50 and U-Net) show modest gains, confirming the broad utility of the MAT framework, while multimodal LLMs uniquely excel at compositional generalization. CrossMed provides a rigorous testbed for evaluating zero-shot, cross-task, and modality-agnostic generalization in medical vision-language models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 24 references that are in the map. First posted on arXiv 14 Nov 2025.

Builds on

15 of 24 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Focus on What Matters: Enhancing Medical Vision-Language Models with Automatic Attention Alignment Tuning24 May
  2. 2024Exploring Compositional Generalization of Multimodal LLMs for Medical Imaging28 Dec
  3. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale27 Jun
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 11

  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  3. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  4. 2018MINE: Mutual Information Neural Estimationcited by 683
  5. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
  6. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  7. 2020MosMedData: Chest CT Scans With COVID-19 Related Findings Datasetcited by 104
  8. 2023Overcoming Data Scarcity in Biomedical Imaging with a Foundational Multi-Task Modelcited by 40
  9. 2023Domain Generalization in Computational Pathology: Survey and Guidelinescited by 29
  10. 2024MultiMed: Massively Multimodal and Multitask Medical Understandingcited by 9
  11. 2024Automatic Differential Diagnosis using Transformer-Based Multi-Label Sequence Classificationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text
  2. 2024Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA
  3. 2024PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imaging
  4. 2025SMMILE: An Expert-Driven Benchmark for Multimodal Medical In-Context Learning
  5. 2025MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation
  6. 2024Simplifying Multimodality: Unimodal Approach to Multimodal Challenges in Radiology with General-Domain Large Language Model
  7. 2025Benchmarking GPT-5 for Zero-Shot Multimodal Medical Reasoning in Radiology and Radiation Oncology
  8. 2025Computed Tomography Visual Question Answering with Cross-modal Feature Graphing