# General Debiasing for Multimodal Sentiment Analysis

arXiv: https://arxiv.org/abs/2307.10511 · HTML: https://mlsanity.com/paper/2307.10511/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 55 references that are in the map.
- First posted on arXiv 20 Jul 2023.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [Efficient Low-rank Multimodal Fusion with Modality-Specific Factors](https://mlsanity.com/paper/1806.00064/) (2018)
- [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2005.03545/) (2020)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos](https://mlsanity.com/paper/1606.06259/) (2016)
- [Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2102.04830/) (2021)
- [Memory Fusion Network for Multi-view Sequential Learning](https://mlsanity.com/paper/1802.00927/) (2018)
- [Learning Robust Representations by Projecting Superficial Statistics Out](https://mlsanity.com/paper/1903.06256/) (2019)
- [Learning Debiased Representation via Disentangled Feature Augmentation](https://mlsanity.com/paper/2107.01372/) (2021)
- [Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors](https://mlsanity.com/paper/1811.09362/) (2018)
- [Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure](https://mlsanity.com/paper/2209.14107/) (2022)
- [Modality to Modality Translation: An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion](https://mlsanity.com/paper/1911.07848/) (2019)
- [Multimodal End-to-End Sparse Model for Emotion Recognition](https://mlsanity.com/paper/2103.09666/) (2021)
- [CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation](https://mlsanity.com/paper/2207.14087/) (2022)
- [User-controllable Recommendation Against Filter Bubbles](https://mlsanity.com/paper/2204.13844/) (2022)
- [Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis](https://mlsanity.com/paper/2207.11652/) (2022)
- [Twitter Sentiment Analysis via Bi-sense Emoji Embedding and Attention-based LSTM](https://mlsanity.com/paper/1807.07961/) (2018)
- [Efficient Graph Deep Learning in TensorFlow with tf_geometric](https://mlsanity.com/paper/2101.11552/) (2021)
- [HS-GCN: Hamming Spatial Graph Convolutional Networks for Recommendation](https://mlsanity.com/paper/2301.05430/) (2023)
- [Class Is Invariant to Context and Vice Versa: On Learning Invariance for Out-Of-Distribution Generalization](https://mlsanity.com/paper/2208.03462/) (2022)

## Built on by (most cited first, 10 of 10)

- [Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitives](https://mlsanity.com/paper/2403.10518/) (2024)
- [Can Large Vision-Language Models Understand Multimodal Sarcasm?](https://mlsanity.com/paper/2508.03654/) (2025)
- [Towards Robust Multimodal Emotion Recognition under Missing Modalities and Distribution Shifts](https://mlsanity.com/paper/2506.10452/) (2025)
- [Out-of-Distribution Detection with Attention Head Masking for Multimodal Document Classification](https://mlsanity.com/paper/2408.11237/) (2024)
- [Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2508.04999/) (2025)
- [Towards Minimal Causal Representations for Human Multimodal Language Understanding](https://mlsanity.com/paper/2509.21805/) (2025)
- [Calibrating Multimodal Consensus for Emotion Recognition](https://mlsanity.com/paper/2510.20256/) (2025)
- [CorMulT: A Semi-supervised Modality Correlation-aware Multimodal Transformer for Sentiment Analysis](https://mlsanity.com/paper/2407.07046/) (2024)
- [Multi-Modal Image Fusion via Intervention-Stable Feature Learning](https://mlsanity.com/paper/2603.23272/) (2026)
- [An Enhanced Dual Transformer Contrastive Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2510.23617/) (2025)

## Abstract (arXiv)

Existing work on Multimodal Sentiment Analysis (MSA) utilizes multimodal information for prediction yet unavoidably suffers from fitting the spurious correlations between multimodal features and sentiment labels. For example, if most videos with a blue background have positive labels in a dataset, the model will rely on such correlations for prediction, while "blue background" is not a sentiment-related feature. To address this problem, we define a general debiasing MSA task, which aims to enhance the Out-Of-Distribution (OOD) generalization ability of MSA models by reducing their reliance on spurious correlations. To this end, we propose a general debiasing framework based on Inverse Probability Weighting (IPW), which adaptively assigns small weights to the samples with larger bias (i.e., the severer spurious correlations). The key to this debiasing framework is to estimate the bias of each sample, which is achieved by two steps: 1) disentangling the robust features and biased features in each modality, and 2) utilizing the biased features to estimate the bias. Finally, we employ IPW to reduce the effects of large-biased samples, facilitating robust feature learning for sentiment prediction. To examine the model's generalization ability, we keep the original testing sets on two benchmarks and additionally construct multiple unimodal and multimodal OOD testing sets. The empirical results demonstrate the superior generalization ability of our proposed framework. We have released the code and data to facilitate the reproduction https://github.com/Teng-Sun/GEAR.