# Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis

arXiv: https://arxiv.org/abs/2508.04999 · HTML: https://mlsanity.com/paper/2508.04999/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 96 references that are in the map.
- First posted on arXiv 7 Aug 2025.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [Efficient Low-rank Multimodal Fusion with Modality-Specific Factors](https://mlsanity.com/paper/1806.00064/) (2018)
- [Counterfactual VQA: A Cause-Effect Look at Language Bias](https://mlsanity.com/paper/2006.04315/) (2020)
- [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2005.03545/) (2020)
- [Discovering Invariant Rationales for Graph Neural Networks](https://mlsanity.com/paper/2201.12872/) (2022)
- [Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2102.04830/) (2021)
- [Memory Fusion Network for Multi-view Sequential Learning](https://mlsanity.com/paper/1802.00927/) (2018)
- [Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2109.00412/) (2021)
- [Causal Intervention for Leveraging Popularity Bias in Recommendation](https://mlsanity.com/paper/2105.06067/) (2021)
- [Causal Attention for Interpretable and Generalizable Graph Classification](https://mlsanity.com/paper/2112.15089/) (2021)
- [Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis](https://mlsanity.com/paper/1911.05544/) (2019)
- [Provable Dynamic Fusion for Low-Quality Multimodal Data](https://mlsanity.com/paper/2306.02050/) (2023)
- [Decoupled Multimodal Distilling for Emotion Recognition](https://mlsanity.com/paper/2303.13802/) (2023)
- [Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering](https://mlsanity.com/paper/2207.12647/) (2022)
- [Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations](https://mlsanity.com/paper/2210.17444/) (2022)
- [Modality to Modality Translation: An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion](https://mlsanity.com/paper/1911.07848/) (2019)
- [Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis](https://mlsanity.com/paper/2107.13669/) (2021)
- [Context De-confounded Emotion Recognition](https://mlsanity.com/paper/2303.11921/) (2023)
- [Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2310.05804/) (2023)
- [Towards Multimodal Sentiment Analysis Debiasing via Bias Purification](https://mlsanity.com/paper/2403.05023/) (2024)
- [DLF: Disentangled-Language-Focused Multimodal Sentiment Analysis](https://mlsanity.com/paper/2412.12225/) (2024)
- [Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Content](https://mlsanity.com/paper/2412.10460/) (2024)
- [Robust Emotion Recognition in Context Debiasing](https://mlsanity.com/paper/2403.05963/) (2024)
- [DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learning](https://mlsanity.com/paper/2503.11892/) (2025)
- [M-SENA: An Integrated Platform for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2203.12441/) (2022)

## Built on by (most cited first, 2 of 2)

- [Multi-Modal Image Fusion via Intervention-Stable Feature Learning](https://mlsanity.com/paper/2603.23272/) (2026)
- [Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysis](https://mlsanity.com/paper/2609.30470/) (2026)

## Abstract (arXiv)

Multimodal sentiment analysis (MSA) aims to understand human emotions by integrating information from multiple modalities, such as text, audio, and visual data. However, existing methods often suffer from spurious correlations both within and across modalities, leading models to rely on statistical shortcuts rather than true causal relationships, thereby undermining generalization. To mitigate this issue, we propose a Multi-relational Multimodal Causal Intervention (MMCI) framework, which leverages the backdoor adjustment from causal theory to address the confounding effects of such shortcuts. Specifically, we first model the multimodal inputs as a multi-relational graph to explicitly capture intra- and inter-modal dependencies. Then, we apply an attention mechanism to separately estimate and disentangle the causal features and shortcut features corresponding to these intra- and inter-modal relations. Finally, by approximating backdoor adjustment, we stratify the shortcut features and dynamically combine them with the causal features to encourage MMCI to produce stable predictions under distribution shifts. Extensive experiments on several standard MSA datasets and out-of-distribution (OOD) settings demonstrate that our method effectively suppresses biases and improves performance.