mlsanityData to 7 Oct 2026

Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysis

arXiv 2410.04491 ↗First posted 6 Oct 2024cs.CL cs.AI cs.MMcited by 7

Not on the map. Why?

Abstract (arXiv)

Multimodal Sentiment Analysis (MSA) utilizes multimodal data to infer the users' sentiment. Previous methods focus on equally treating the contribution of each modality or statically using text as the dominant modality to conduct interaction, which neglects the situation where each modality may become dominant. In this paper, we propose a Knowledge-Guided Dynamic Modality Attention Fusion Framework (KuDA) for multimodal sentiment analysis. KuDA uses sentiment knowledge to guide the model dynamically selecting the dominant modality and adjusting the contributions of each modality. In addition, with the obtained multimodal representation, the model can further highlight the contribution of dominant modality through the correlation evaluation loss. Extensive experiments on four MSA benchmark datasets indicate that KuDA achieves state-of-the-art performance and is able to adapt to different scenarios of dominant modality.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 16 of its 28 references that are in the map. First posted on arXiv 6 Oct 2024.

Builds on

16 of 28 references.

References · 16

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  5. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  6. 2018Efficient Low-rank Multimodal Fusion with Modality-Specific Factorscited by 268
  7. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 262
  8. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 179
  9. 2018Memory Fusion Network for Multi-view Sequential Learningcited by 167
  10. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 130
  11. 2020A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysiscited by 46
  12. 2021Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysiscited by 45
4 more
  1. 2022Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Modulecited by 43
  2. 2022CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimationcited by 39
  3. 2021Knowledge Enhanced Pretrained Language Models: A Compreshensive Surveycited by 33
  4. 2021Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysiscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Towards Minimal Causal Representations for Human Multimodal Language Understandingcited by 2
  2. 2025Uncertainty-Aware Collaborative System of Large and Small Models for Multimodal Sentiment Analysiscited by 1
  3. 2026CaReFlow: Cyclic Adaptive Rectified Flow for Multimodal Fusioncited by 1
  4. 2025Multi-Modal Opinion Integration for Financial Sentiment Analysis using Cross-Modal Attentioncited by 0
  5. 2026GRCF: Two-Stage Groupwise Ranking and Calibration Framework for Multimodal Sentiment Analysiscited by 0
  6. 2026Segregate, Refine, Integrate: Decomposing Multimodal Fusion for Sentiment Analysiscited by 0
  7. 2026Multi-Granularity Sentiment Integration for LLM-Based Multimodal Sentiment Analysiscited by 0