mlsanityData to 14 Oct 2026

Multimodal Causal-Driven Representation Learning for Generalizable Medical Image Segmentation

arXiv 2508.05008 ↗First posted 7 Aug 2025cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision-Language Models (VLMs), such as CLIP, have demonstrated remarkable zero-shot capabilities in various computer vision tasks. However, their application to medical imaging remains challenging due to the high variability and complexity of medical data. Specifically, medical images often exhibit significant domain shifts caused by various confounders, including equipment differences, procedure artifacts, and imaging modes, which can lead to poor generalization when models are applied to unseen domains. To address this limitation, we propose Multimodal Causal-Driven Representation Learning (MCDRL), a novel framework that integrates causal inference with the VLM to tackle domain generalization in medical image segmentation. MCDRL is implemented in two steps: first, it leverages CLIP's cross-modal capabilities to identify candidate lesion regions and construct a confounder dictionary through text prompts, specifically designed to represent domain-specific variations; second, it trains a causal intervention network that utilizes this dictionary to identify and eliminate the influence of these domain-specific variations while preserving the anatomical structural information critical for segmentation tasks. Extensive experiments demonstrate that MCDRL consistently outperforms competing methods, yielding superior segmentation accuracy and exhibiting robust generalizability.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 52 references that are in the map. First posted on arXiv 7 Aug 2025.

Builds on

35 of 52 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  5. 2019Invariant Risk Minimizationcited by 1,854
  6. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  7. 2020In Search of Lost Domain Generalizationcited by 1,054
  8. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  9. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
  10. 2022MaPLe: Multi-modal Prompt Learningcited by 687
  11. 2013Domain Generalization via Invariant Feature Representationcited by 623
  12. 2019Kvasir-SEG: A Segmented Polyp Datasetcited by 589
23 more
  1. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 450
  2. 2023LangSplat: 3D Language Gaussian Splattingcited by 415
  3. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 407
  4. 2021Domain Adaptation for Medical Image Analysis: A Surveycited by 352
  5. 2024What matters when building vision-language models?cited by 326
  6. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 323
  7. 2020Domain Generalization using Causal Matchingcited by 225
  8. 2020A Survey on Causal Inferencecited by 202
  9. 2022MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretrainingcited by 187
  10. 2020Efficient Domain Generalization via Common-Specific Low-Rank Decompositioncited by 134
  11. 2016Unsupervised domain adaptation in brain lesion segmentation with adversarial networkscited by 125
  12. 2020Learning from Extrinsic and Intrinsic Supervisions for Domain Generalizationcited by 108
  13. 2020Towards Universal Representation Learning for Deep Face Recognitioncited by 78
  14. 2020A causal view of compositional zero-shot recognitioncited by 78
  15. 2023Domain Adaptation for Time Series Under Feature and Label Shiftscited by 60
  16. 2018A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocolscited by 49
  17. 2023Leveraging Vision-Language Models for Improving Domain Generalization in Image Classificationcited by 41
  18. 2024MoDE: CLIP Data Experts via Clusteringcited by 22
  19. 2022TempCLR: Temporal Alignment Representation with Contrastive Learningcited by 15
  20. 2023Parameter Efficient Fine-tuning via Cross Block Orchestration for Segment Anything Modelcited by 15
  21. 2023SurgPLAN: Surgical Phase Localization Network for Phase Recognitioncited by 12
  22. 2025MAP: Evaluation and Multi-Agent Enhancement of Large Language Models for Inpatient Pathwayscited by 10
  23. 2025Text-Driven Causal Representation Learning for Source-Free Domain Generalizationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026DVLA-RL: Dual-Level Vision-Language Alignment with Reinforcement Learning Gating for Few-Shot Learningcited by 2
  2. 2026Causal Transfer in Medical Image Analysiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Generalizable Single-Source Cross-modality Medical Image Segmentation via Invariant Causal Mechanisms
  2. 2025CausalCLIPSeg: Unlocking CLIP's Potential in Referring Medical Image Segmentation with Causal Intervention
  3. 2025Causal-SAM-LLM: Large Language Models as Causal Reasoners for Robust Medical Segmentation
  4. 2026Leveraging Causal Reasoning Method for Explaining Medical Image Segmentation Models
  5. 2026MedCLIPSeg: Probabilistic Vision-Language Adaptation for Data-Efficient and Generalizable Medical Image Segmentation
  6. 2026Characterizing Text Branch Sensitivity in Medical Vision-Language Segmentation via Evidence Decoupling
  7. 2026Making medical vision-language models think causally across modalities with retrieval-augmented cross-modal reasoning
  8. 2026SegBanana: Steering Unified Multimodal Models into Medical Segmenters