Dual Causal Inference: Integrating Backdoor Adjustment and Instrumental Variable Learning for Medical VQA
Not on the map. Why?
Abstract (arXiv)
Medical Visual Question Answering (MedVQA) aims to generate clinically reliable answers conditioned on complex medical images and questions. However, existing methods often overfit to superficial cross-modal correlations, neglecting the intrinsic biases embedded in multimodal medical data. Consequently, models become vulnerable to cross-modal confounding effects, severely hindering their ability to provide trustworthy diagnostic reasoning. To address this limitation, we propose a novel Dual Causal Inference (DCI) framework for MedVQA. To the best of our knowledge, DCI is the first unified architecture that integrates Backdoor Adjustment (BDA) and Instrumental Variable (IV) learning to jointly tackle both observable and unobserved confounders. Specifically, we formulate a Structural Causal Model (SCM) where observable cross-modal biases (e.g., frequent visual and textual co-occurrences) are mitigated via BDA, while unobserved confounders are compensated using an IV learned from a shared latent space. To guarantee the validity of the IV, we design mutual information constraints that maximize its dependence on the fused multimodal representations while minimizing its associations with the unobserved confounders and target answers. Through this dual mechanism, DCI extracts deconfounded representations that capture genuine causal relationships. Extensive experiments on four benchmark datasets, SLAKE, SLAKE-CP, VQA-RAD, and PathVQA, demonstrate that our method consistently outperforms existing approaches, particularly in out-of-distribution (OOD) generalization. Furthermore, qualitative analyses confirm that DCI significantly enhances the interpretability and robustness of cross-modal reasoning by explicitly disentangling true causal effects from spurious cross-modal shortcuts.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 45 references that are in the map. First posted on arXiv 22 Apr 2026.
Builds on
27 of 45 references.
References · 27
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
- 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
- 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
- 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
- 2020Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effectcited by 256
- 2020CLUB: A Contrastive Log-ratio Upper Bound of Mutual Informationcited by 233
- 2020Causal Intervention for Weakly-Supervised Semantic Segmentationcited by 229
- 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
- 2022From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Modelscited by 114
- 2021Causal Attention for Vision-Language Taskscited by 111
15 more
- 2022Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Trainingcited by 104
- 2019Overcoming Data Limitation in Medical Visual Question Answeringcited by 71
- 2023Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Modelscited by 58
- 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
- 2022Self-supervised vision-language pretraining for Medical visual question answeringcited by 30
- 2023Masked Vision and Language Pre-training with Unimodal and Multimodal Contrastive Losses for Medical Visual Question Answeringcited by 29
- 2023Cross-Modal Causal Intervention for Medical Report Generationcited by 14
- 2024MISS: A Generative Pretraining and Finetuning Approach for Med-VQAcited by 14
- 2022Instrumental Variables in Causal Inference and Machine Learning: A Surveycited by 10
- 2023Chest X-ray Image Classification: A Causal Perspectivecited by 8
- 2021Instrumental Variable-Driven Domain Generalization with Unobserved Confounderscited by 7
- 2022A Dual-Attention Learning Network with Word and Sentence Embedding for Medical Visual Question Answeringcited by 5
- 2023Localized Questions in Medical Visual Question Answeringcited by 5
- 2025MOTOR: Multimodal Optimal Transport via Grounded Retrieval in Medical Visual Question Answeringcited by 5
- 2023Instrumental Variable Learning for Chest X-ray Classificationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.