Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysis
Not on the map. Why?
Abstract (arXiv)
Multimodal Sentiment Analysis (MSA) aims to infer human emotions from multiple modalities such as text, audio, and vision. In practice, inputs are often corrupted by noise and missing modalities, which degrades performance. Existing methods typically address these challenges in isolation, limiting their effectiveness in realistic settings. To address this limitation, we propose a Reliability-aware Cross-sample Enhancement (RCE) framework. Specifically, RCE first introduces an adaptive variational information bottleneck to model modality-wise uncertainty and perform quality-aware information compression, thereby suppressing redundant noise in unreliable modalities. Furthermore, we design a reliability-aware cross-sample enhancement strategy that retrieves high-confidence, semantically consistent neighbors from a large candidate pool to enrich and calibrate current representations, effectively alleviating information deficiency caused by missing modalities. Building upon this, RCE integrates cross-modal interactions with a multilevel reliability-aware fusion mechanism to adaptively aggregate information across modalities and enhancement stages, leading to more robust multimodal representations. Extensive experiments demonstrate that RCE consistently outperforms state-of-the-art methods across full, noisy, and missing-modality settings.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 70 references that are in the map. First posted on arXiv 24 Sep 2026.
Builds on
44 of 70 references.
References · 44
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
- 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
- 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
- 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 179
- 2019Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)cited by 173
- 2018Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalitiescited by 149
- 2019UR-FUNNY: A Multimodal Language Dataset for Understanding Humorcited by 130
- 2022GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversationcited by 86
- 2023Provable Dynamic Fusion for Low-Quality Multimodal Datacited by 81
32 more
- 2023Decoupled Multimodal Distilling for Emotion Recognitioncited by 76
- 2023Boosting Multi-modal Model Performance with Adaptive Gradient Modulationcited by 71
- 2022Dynamic Multimodal Fusioncited by 54
- 2022Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representationscited by 48
- 2021Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysiscited by 45
- 2024Probabilistic Contrastive Learning for Long-Tailed Visual Recognitioncited by 42
- 2021Multimodal Learning using Optimal Transport for Sarcasm and Humor Detectioncited by 36
- 2024Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 32
- 2019Learning Representations from Imperfect Time Series Data via Tensor Rank Regularizationcited by 30
- 2023Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysiscited by 27
- 2024DLF: Disentangled-Language-Focused Multimodal Sentiment Analysiscited by 27
- 2024Towards Robust Multimodal Sentiment Analysis with Incomplete Datacited by 26
- 2024Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Contentcited by 25
- 2025DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learningcited by 23
- 2019Privacy Enhanced Multimodal Neural Representations for Emotion Recognitioncited by 22
- 2022Analyzing Modality Robustness in Multimodal Sentiment Analysiscited by 20
- 2024Neuro-Inspired Information-Theoretic Hierarchical Perception for Multimodal Learningcited by 18
- 2024Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learningcited by 16
- 2025Learning Optimal Multimodal Information Bottleneck Representationscited by 15
- 2024DepMamba: Progressive Fusion Mamba for Multimodal Depression Detectioncited by 14
- 2024Debiased Multimodal Understanding for Human Language Sequencescited by 10
- 2025Improving Multimodal Sentiment Analysis via Modality Optimization and Dynamic Primary Modality Selectioncited by 6
- 2022Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysiscited by 5
- 2026CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learningcited by 4
- 2025Towards Explainable Fusion and Balanced Learning in Multimodal Sentiment Analysiscited by 3
- 2025TMDC: A Two-Stage Modality Denoising and Complementation Framework for Multimodal Sentiment Analysis with Missing and Noisy Modalitiescited by 3
- 2025Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysiscited by 2
- 2025Towards Minimal Causal Representations for Human Multimodal Language Understandingcited by 2
- 2025PaSE: Prototype-aligned Calibration and Shapley-based Equilibrium for Multimodal Sentiment Analysiscited by 2
- 2025PSA-MF: Personality-Sentiment Aligned Multi-Level Fusion for Multimodal Sentiment Analysiscited by 1
- 2026Decoupled Hierarchical Distillation for Multimodal Emotion Recognitioncited by 1
- 2026CaReFlow: Cyclic Adaptive Rectified Flow for Multimodal Fusioncited by 1
Built on by
Not cited yet.