mlsanityData to 14 Oct 2026

Analyzing Modality Robustness in Multimodal Sentiment Analysis

arXiv 2205.15465 ↗First posted 30 May 2022cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Building robust multimodal models are crucial for achieving reliable deployment in the wild. Despite its importance, less attention has been paid to identifying and improving the robustness of Multimodal Sentiment Analysis (MSA) models. In this work, we hope to address that by (i) Proposing simple diagnostic checks for modality robustness in a trained multimodal model. Using these checks, we find MSA models to be highly sensitive to a single modality, which creates issues in their robustness; (ii) We analyze well-known robust training strategies to alleviate the issues. Critically, we observe that robustness can be achieved without compromising on the original performance. We hope our extensive study-performed across five models and two benchmark datasets-and proposed procedures would make robustness an integral component in MSA research. Our diagnostic checks and robust training solutions are simple to implement and available at https://github. com/declare-lab/MSA-Robustness.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 17 of its 22 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

17 of 22 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  3. 2018Robustness May Be at Odds with Accuracycited by 880
  4. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
  5. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
  6. 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 245
  7. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 180
  8. 2018Learning Factorized Multimodal Representationscited by 171
  9. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  10. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 131
  11. 2021Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformerscited by 72
  12. 2020Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropiescited by 57
5 more
  1. 2021Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysiscited by 45
  2. 2019Adversarial Robustness May Be at Odds With Simplicitycited by 39
  3. 2022Adaptive Memory Networks with Self-supervised Learning for Unsupervised Anomaly Detectioncited by 18
  4. 2021Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysiscited by 5
  5. 2018Adaptive Memory Networkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 6; 2025: 2; 2026: 5.

20 papers.

Sort:

By area:

  1. 2023MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learningcited by 33
  2. 2023Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptationcited by 29
  3. 2024MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognitioncited by 23
  4. 2022Efficient Multimodal Transformer with Dual-Level Feature Restoration for Robust Multimodal Sentiment Analysiscited by 20
  5. 2024Promoting cross-modal representations to improve multimodal foundation models for physiological signalscited by 13
  6. 2022SANCL: Multimodal Review Helpfulness Prediction with Selective Attention and Natural Contrastive Learningcited by 12
  7. 2024A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognitioncited by 12
  8. 2024MMP: Towards Robust Multi-Modal Learning with Masked Modality Projectioncited by 10
  9. 2024Trustworthy Multimodal Fusion for Sentiment Analysis in Ordinal Sentiment Spacecited by 8
  10. 2024Leveraging Contrastive Learning and Self-Training for Multimodal Emotion Recognition with Limited Labeled Samplescited by 5
  11. 2026CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learningcited by 4
  12. 2025DeepMLF: Multimodal language model with learnable tokens for deep fusion in sentiment analysiscited by 3
8 more
  1. 2022Adapted Multimodal BERT with Layer-wise Fusion for Sentiment Analysiscited by 2
  2. 2025Robust Multimodal Learning via Cross-Modal Proxy Tokenscited by 2
  3. 2026Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflectioncited by 2
  4. 2023PowMix: A Versatile Regularizer for Multimodal Sentiment Analysiscited by 1
  5. 2026Addressing Missing and Noisy Modalities in One Solution: Unified Modality-Quality Framework for Low-quality Multimodal Datacited by 1
  6. 2023Modality Influence in Multimodal Machine Learningcited by 0
  7. 2026The Illusion of Balanced Multimodal Sentiment Analysis: Beyond the Limits of Optimization-Based Methodscited by 0
  8. 2026Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysiscited by 0