mlsanityData to 7 Oct 2026

Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysis

arXiv 2609.30470 ↗First posted 24 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal Sentiment Analysis (MSA) aims to infer human emotions from multiple modalities such as text, audio, and vision. In practice, inputs are often corrupted by noise and missing modalities, which degrades performance. Existing methods typically address these challenges in isolation, limiting their effectiveness in realistic settings. To address this limitation, we propose a Reliability-aware Cross-sample Enhancement (RCE) framework. Specifically, RCE first introduces an adaptive variational information bottleneck to model modality-wise uncertainty and perform quality-aware information compression, thereby suppressing redundant noise in unreliable modalities. Furthermore, we design a reliability-aware cross-sample enhancement strategy that retrieves high-confidence, semantically consistent neighbors from a large candidate pool to enrich and calibrate current representations, effectively alleviating information deficiency caused by missing modalities. Building upon this, RCE integrates cross-modal interactions with a multilevel reliability-aware fusion mechanism to adaptively aggregate information across modalities and enhancement stages, leading to more robust multimodal representations. Extensive experiments demonstrate that RCE consistently outperforms state-of-the-art methods across full, noisy, and missing-modality settings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 70 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

44 of 70 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  4. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  5. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  6. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  7. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 179
  8. 2019Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)cited by 173
  9. 2018Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalitiescited by 149
  10. 2019UR-FUNNY: A Multimodal Language Dataset for Understanding Humorcited by 130
  11. 2022GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversationcited by 86
  12. 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 81
32 more
  1. 2023Decoupled Multimodal Distilling for Emotion Recognitioncited by 76
  2. 2023Boosting Multi-modal Model Performance with Adaptive Gradient Modulationcited by 71
  3. 2022Dynamic Multimodal Fusioncited by 54
  4. 2022Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representationscited by 48
  5. 2021Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysiscited by 45
  6. 2024Probabilistic Contrastive Learning for Long-Tailed Visual Recognitioncited by 42
  7. 2021Multimodal Learning using Optimal Transport for Sarcasm and Humor Detectioncited by 36
  8. 2024Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 32
  9. 2019Learning Representations from Imperfect Time Series Data via Tensor Rank Regularizationcited by 30
  10. 2023Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysiscited by 27
  11. 2024DLF: Disentangled-Language-Focused Multimodal Sentiment Analysiscited by 27
  12. 2024Towards Robust Multimodal Sentiment Analysis with Incomplete Datacited by 26
  13. 2024Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Contentcited by 25
  14. 2025DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learningcited by 23
  15. 2019Privacy Enhanced Multimodal Neural Representations for Emotion Recognitioncited by 22
  16. 2022Analyzing Modality Robustness in Multimodal Sentiment Analysiscited by 20
  17. 2024Neuro-Inspired Information-Theoretic Hierarchical Perception for Multimodal Learningcited by 18
  18. 2024Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learningcited by 16
  19. 2025Learning Optimal Multimodal Information Bottleneck Representationscited by 15
  20. 2024DepMamba: Progressive Fusion Mamba for Multimodal Depression Detectioncited by 14
  21. 2024Debiased Multimodal Understanding for Human Language Sequencescited by 10
  22. 2025Improving Multimodal Sentiment Analysis via Modality Optimization and Dynamic Primary Modality Selectioncited by 6
  23. 2022Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysiscited by 5
  24. 2026CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learningcited by 4
  25. 2025Towards Explainable Fusion and Balanced Learning in Multimodal Sentiment Analysiscited by 3
  26. 2025TMDC: A Two-Stage Modality Denoising and Complementation Framework for Multimodal Sentiment Analysis with Missing and Noisy Modalitiescited by 3
  27. 2025Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysiscited by 2
  28. 2025Towards Minimal Causal Representations for Human Multimodal Language Understandingcited by 2
  29. 2025PaSE: Prototype-aligned Calibration and Shapley-based Equilibrium for Multimodal Sentiment Analysiscited by 2
  30. 2025PSA-MF: Personality-Sentiment Aligned Multi-Level Fusion for Multimodal Sentiment Analysiscited by 1
  31. 2026Decoupled Hierarchical Distillation for Multimodal Emotion Recognitioncited by 1
  32. 2026CaReFlow: Cyclic Adaptive Rectified Flow for Multimodal Fusioncited by 1

Built on by

Not cited yet.