mlsanityData to 14 Oct 2026

Should Missing Modalities Always Be Necessary to Repair for Multi-modal Sentiment Analysis?

arXiv 2607.17262 ↗First posted 19 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Existing methods for multimodal sentiment analysis (MSA) under missing modalities usually follow a repair-first paradigm. We revisit this assumption and ask: should every missing modality be repaired? A per-sample oracle analysis shows the answer is not always: full-modality input is optimal for only a small fraction of samples, and every modality subset is preferred by some samples. These results suggest that adding or repairing modalities may not always improve prediction, and that the utility of each modality is sample-dependent. Building on this finding, we propose Sufficiency-Informed Evidential ValvE (SIEVE) that turns ``whether to repair'' into an explicit, learnable decision at the sample level. SIEVE compares a direct prediction branch with a repair branch, derives an empirical sufficiency signal from their per-sample loss gap, and routes each input through an evidential gate that jointly models sufficiency and its epistemic uncertainty. SIEVE is repair-agnostic: it operates as a plug-and-play decision on top of any explicit or implicit repair module, without modifying its internal design. Experiments on CMU-MOSI and IEMOCAP show that SIEVE consistently improves representative repair backbones across evaluated missing rates, and approaches the per-sample dual-branch achievable optimum.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 36 references that are in the map. First posted on arXiv 19 Jul 2026.

Builds on

19 of 36 references.

References · 19

  1. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  2. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  3. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 903
  4. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 732
  5. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
  6. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 433
  7. 2019Deep Evidential Regressioncited by 355
  8. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
  9. 2016MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videoscited by 194
  10. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 180
  11. 2022Trusted Multi-View Classification with Dynamic Evidential Fusioncited by 168
  12. 2018Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalitiescited by 149
7 more
  1. 2021Trusted Multi-View Classificationcited by 94
  2. 2022GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversationcited by 87
  3. 2023MMANet: Margin-aware Distillation and Modality-aware Regularization for Incomplete Multimodal Learningcited by 40
  4. 2024Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 32
  5. 2024Towards Robust Multimodal Sentiment Analysis with Incomplete Datacited by 27
  6. 2024Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learningcited by 16
  7. 2026Evaluation Before Generation: A Paradigm for Robust Multimodal Sentiment Analysis with Missing Modalitiescited by 1

Built on by

Not cited yet.