mlsanityData to 14 Oct 2026

Towards Explainable Fusion and Balanced Learning in Multimodal Sentiment Analysis

arXiv 2504.12151 ↗First posted 16 Apr 2025cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Multimodal Sentiment Analysis (MSA) faces two critical challenges: the lack of interpretability in the decision logic of multimodal fusion and modality imbalance caused by disparities in inter-modal information density. To address these issues, we propose KAN-MCP, a novel framework that integrates the interpretability of Kolmogorov-Arnold Networks (KAN) with the robustness of the Multimodal Clean Pareto (MCPareto) framework. First, KAN leverages its univariate function decomposition to achieve transparent analysis of cross-modal interactions. This structural design allows direct inspection of feature transformations without relying on external interpretation tools, thereby ensuring both high expressiveness and interpretability. Second, the proposed MCPareto enhances robustness by addressing modality imbalance and noise interference. Specifically, we introduce the Dimensionality Reduction and Denoising Modal Information Bottleneck (DRD-MIB) method, which jointly denoises and reduces feature dimensionality. This approach provides KAN with discriminative low-dimensional inputs to reduce the modeling complexity of KAN while preserving critical sentiment-related information. Furthermore, MCPareto dynamically balances gradient contributions across modalities using the purified features output by DRD-MIB, ensuring lossless transmission of auxiliary signals and effectively alleviating modality imbalance. This synergy of interpretability and robustness not only achieves superior performance on benchmark datasets such as CMU-MOSI, CMU-MOSEI, and CH-SIMS v2 but also offers an intuitive visualization interface through KAN's interpretable architecture. Our code is released on https://github.com/LuoMSen/KAN-MCP.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 26 of its 74 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

26 of 74 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  5. 2016Deep Variational Information Bottleneckcited by 930
  6. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
  7. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 546
  8. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 433
  9. 2018Efficient Low-rank Multimodal Fusion with Modality-Specific Factorscited by 270
  10. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
  11. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 264
  12. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 180
14 more
  1. 2022SAITS: Self-Attention-based Imputation for Time Seriescited by 169
  2. 2018Memory Fusion Network for Multi-view Sequential Learningcited by 168
  3. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156
  4. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 131
  5. 2020Explaining the Explainer: A First Theoretical Analysis of LIMEcited by 89
  6. 2024MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistancecited by 73
  7. 2020CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualizationcited by 57
  8. 2022Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representationscited by 48
  9. 2022Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Modulecited by 44
  10. 2024Classifier-guided Gradient Modulation for Enhanced Multimodal Learningcited by 35
  11. 2022Abs-CAM: A Gradient Optimization Interpretable Approach for Explanation of Convolutional Neural Networkscited by 20
  12. 2024Neuro-Inspired Information-Theoretic Hierarchical Perception for Multimodal Learningcited by 18
  13. 2021Quantum-inspired Multimodal Fusion for Video Sentiment Analysiscited by 13
  14. 2023Multimodal Sentiment Analysis: A Surveycited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities
  2. 2026QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis
  3. 2026AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba
  4. 2025Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis
  5. 2025Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalities
  6. 2025BTW: A Non-Parametric Variance Stabilization Framework for Multimodal Model Integration
  7. 2025FSRF: Factorization-guided Semantic Recovery for Incomplete Multimodal Sentiment Analysis
  8. 2026CAGMamba: Context-Aware Gated Cross-Modal Mamba Network for Multimodal Sentiment Analysis