mlsanityData to 7 Oct 2026

Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Content

arXiv 2412.10460 ↗First posted 12 Dec 2024cs.CV cs.AI cs.SDcited by 25

Not on the map. Why?

Abstract (arXiv)

Multimodal Sentiment Analysis (MSA) stands as a critical research frontier, seeking to comprehensively unravel human emotions by amalgamating text, audio, and visual data. Yet, discerning subtle emotional nuances within audio and video expressions poses a formidable challenge, particularly when emotional polarities across various segments appear similar. In this paper, our objective is to spotlight emotion-relevant attributes of audio and visual modalities to facilitate multimodal fusion in the context of nuanced emotional shifts in visual-audio scenarios. To this end, we introduce DEVA, a progressive fusion framework founded on textual sentiment descriptions aimed at accentuating emotional features of visual-audio content. DEVA employs an Emotional Description Generator (EDG) to transmute raw audio and visual data into textualized sentiment descriptions, thereby amplifying their emotional characteristics. These descriptions are then integrated with the source data to yield richer, enhanced features. Furthermore, DEVA incorporates the Text-guided Progressive Fusion Module (TPF), leveraging varying levels of text as a core modality guide. This module progressively fuses visual-audio minor modalities to alleviate disparities between text and visual-audio modalities. Experimental results on widely used sentiment analysis benchmark datasets, including MOSI, MOSEI, and CH-SIMS, underscore significant enhancements compared to state-of-the-art models. Moreover, fine-grained emotion experiments corroborate the robust sensitivity of DEVA to subtle emotional variations.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 1 of its 41 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

1 of 41 references.

References · 1

  1. 2016MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videoscited by 193

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 19.

25 papers.

Sort:

By area:

  1. 2026Tri-Subspaces Disentanglement for Multimodal Sentiment Analysiscited by 5
  2. 2026CLCR: Cross-Level Semantic Collaborative Representation for Multimodal Learningcited by 5
  3. 2026Temporal-Spatial Decouple before Act: Disentangled Representation Learning for Multimodal Sentiment Analysiscited by 3
  4. 2026Group Cognition Learning: Making Everything Better Through Governed Two-Stage Agents Collaborationcited by 3
  5. 2025Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysiscited by 2
  6. 2025Towards Minimal Causal Representations for Human Multimodal Language Understandingcited by 2
  7. 2025Uncertainty-Aware Collaborative System of Large and Small Models for Multimodal Sentiment Analysiscited by 1
  8. 2026Decoupled Hierarchical Distillation for Multimodal Emotion Recognitioncited by 1
  9. 2026CaReFlow: Cyclic Adaptive Rectified Flow for Multimodal Fusioncited by 1
  10. 2026Addressing Missing and Noisy Modalities in One Solution: Unified Modality-Quality Framework for Low-quality Multimodal Datacited by 1
  11. 2026Progressive Representation Learning for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 1
  12. 2026Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysiscited by 1
13 more
  1. 2025Active Multimodal Distillation for Few-shot Action Recognitioncited by 0
  2. 2025QiNN-QJ: A Quantum-inspired Neural Network with Quantum Jump for Multimodal Sentiment Analysiscited by 0
  3. 2025Text-Routed Sparse Mixture-of-Experts Model with Explanation and Temporal Alignment for Multi-Modal Sentiment Analysiscited by 0
  4. 2026GRCF: Two-Stage Groupwise Ranking and Calibration Framework for Multimodal Sentiment Analysiscited by 0
  5. 2026GRAMformer: Any-Order Modality Interactions via Volumetric Multimodal Cross-Attentioncited by 0
  6. 2026Audio Sentiment Analysis via Distillation and Cross-Modal Integration of Generated Multilingual Transcriptscited by 0
  7. 2026Segregate, Refine, Integrate: Decomposing Multimodal Fusion for Sentiment Analysiscited by 0
  8. 2026ProFocus: Interpreting Affective Experience in Artistic Images with Progressive Visual Focusingcited by 0
  9. 2026Multi-Granularity Sentiment Integration for LLM-Based Multimodal Sentiment Analysiscited by 0
  10. 2026Vision-Guided Text Prompt Tuning for Multimodal Sentiment Analysiscited by 0
  11. 2026BiCFlow-MER: Orchestrating Discriminative and Generative Multimodal Emotion Recognition via Conditional Transportcited by 0
  12. 2026SemMSA: Latent Semantic-Aided Robust Multimodal Sentiment Analysis with Incomplete Datacited by 0
  13. 2026Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysiscited by 0