mlsanityData to 7 Oct 2026

Towards Multimodal Sentiment Analysis Debiasing via Bias Purification

arXiv 2403.05023 ↗First posted 8 Mar 2024cs.CL cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

Multimodal Sentiment Analysis (MSA) aims to understand human intentions by integrating emotion-related clues from diverse modalities, such as visual, language, and audio. Unfortunately, the current MSA task invariably suffers from unplanned dataset biases, particularly multimodal utterance-level label bias and word-level context bias. These harmful biases potentially mislead models to focus on statistical shortcuts and spurious correlations, causing severe performance bottlenecks. To alleviate these issues, we present a Multimodal Counterfactual Inference Sentiment (MCIS) analysis framework based on causality rather than conventional likelihood. Concretely, we first formulate a causal graph to discover harmful biases from already-trained vanilla models. In the inference phase, given a factual multimodal input, MCIS imagines two counterfactual scenarios to purify and mitigate these biases. Then, MCIS can make unbiased decisions from biased observations by comparing factual and counterfactual outcomes. We conduct extensive experiments on several standard MSA benchmarks. Qualitative and quantitative results show the effectiveness of the proposed framework.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 25 of its 62 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

25 of 62 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  3. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  4. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
  5. 2018Efficient Low-rank Multimodal Fusion with Modality-Specific Factorscited by 268
  6. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
  7. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 262
  8. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 179
  9. 2018Learning Factorized Multimodal Representationscited by 171
  10. 2018Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalitiescited by 149
  11. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 130
  12. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
13 more
  1. 2020Visual Commonsense R-CNNcited by 106
  2. 2019Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysiscited by 82
  3. 2023Decoupled Multimodal Distilling for Emotion Recognitioncited by 76
  4. 2023Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Modelscited by 59
  5. 2022CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimationcited by 39
  6. 2023AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perceptioncited by 39
  7. 2023Context De-confounded Emotion Recognitioncited by 34
  8. 2022Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysiscited by 18
  9. 2024MISS: A Generative Pretraining and Finetuning Approach for Med-VQAcited by 14
  10. 2023Text-oriented Modality Reinforcement Network for Multimodal Sentiment Analysis from Unaligned Multimodal Sequencescited by 13
  11. 2022Causal Intervention for Subject-Deconfounded Facial Action Unit Recognitioncited by 11
  12. 2023Towards Simultaneous Segmentation of Liver Tumors and Intrahepatic Vessels via Cross-attention Mechanismcited by 8
  13. 2015Don't Just Listen, Use Your Imagination: Leveraging Visual Common Sense for Non-Visual Taskscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 12; 2026: 6.

26 papers.

Sort:

By area:

  1. 2024Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 32
  2. 2025DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learningcited by 23
  3. 2024Networking Systems for Video Anomaly Detection: A Tutorial and Surveycited by 21
  4. 2025Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understandingcited by 18
  5. 2024Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learningcited by 16
  6. 2024De-confounded Data-free Knowledge Distillation for Handling Distribution Shiftscited by 11
  7. 2024Efficiency in Focus: LayerNorm as a Catalyst for Fine-tuning Medical Visual Language Pre-trained Modelscited by 11
  8. 2024Towards Context-Aware Emotion Recognition Debiasing from a Causal Demystification Perspective via De-confounded Trainingcited by 11
  9. 2024Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representationscited by 10
  10. 2025SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Modelcited by 10
  11. 2025Improving Multimodal Sentiment Analysis via Modality Optimization and Dynamic Primary Modality Selectioncited by 6
  12. 2025Beyond Spurious Signals: Debiasing Multimodal Large Language Models via Counterfactual Inference and Adaptive Expert Routingcited by 5
14 more
  1. 2025Towards Robust Multimodal Emotion Recognition under Missing Modalities and Distribution Shiftscited by 4
  2. 2025TiCAL:Typicality-Based Consistency-Aware Learning for Multimodal Emotion Recognitioncited by 4
  3. 2026Temporal-Spatial Decouple before Act: Disentangled Representation Learning for Multimodal Sentiment Analysiscited by 3
  4. 2026Group Cognition Learning: Making Everything Better Through Governed Two-Stage Agents Collaborationcited by 3
  5. 2025Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysiscited by 2
  6. 2025Towards Minimal Causal Representations for Human Multimodal Language Understandingcited by 2
  7. 2024More is Less? A Simulation-Based Approach to Dynamic Interactions between Biases in Multimodal Modelscited by 1
  8. 2025Calibrating Multimodal Consensus for Emotion Recognitioncited by 1
  9. 2026CaReFlow: Cyclic Adaptive Rectified Flow for Multimodal Fusioncited by 1
  10. 2026Addressing Missing and Noisy Modalities in One Solution: Unified Modality-Quality Framework for Low-quality Multimodal Datacited by 1
  11. 2026Progressive Representation Learning for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 1
  12. 2025Weakly Semi-supervised Whole Slide Image Classification by Two-level Cross Consistency Supervisioncited by 0
  13. 2025Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 0
  14. 2026Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspectivecited by 0