mlsanityData to 14 Oct 2026

Multimodal Mixture of Low-Rank Experts for Sentiment Analysis and Emotion Recognition

arXiv 2505.14143 ↗First posted 20 May 2025cs.AIcited by 0

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

Multi-task learning (MTL) enables the efficient transfer of extra knowledge acquired from other tasks. The high correlation between multimodal sentiment analysis (MSA) and multimodal emotion recognition (MER) supports their joint training. However, existing methods primarily employ hard parameter sharing, ignoring parameter conflicts caused by complex task correlations. In this paper, we present a novel MTL method for MSA and MER, termed Multimodal Mixture of Low-Rank Experts (MMoLRE). MMoLRE utilizes shared and task-specific experts to distinctly model common and unique task characteristics, thereby avoiding parameter conflicts. Additionally, inspired by low-rank structures in the Mixture of Experts (MoE) framework, we design low-rank expert networks to reduce parameter and computational overhead as the number of experts increases. Extensive experiments on the CMU-MOSI and CMU-MOSEI benchmarks demonstrate that MMoLRE achieves state-of-the-art performance on the MSA task and competitive results on the MER task.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 19 of its 28 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

19 of 28 references.

References · 19

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  3. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  4. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
  5. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  6. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
  7. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 180
  8. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 131
  9. 2022UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognitioncited by 61
  10. 2024Multi-Task Dense Prediction via Mixture of Low-Rank Expertscited by 58
  11. 2020A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysiscited by 46
  12. 2021Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysiscited by 41
7 more
  1. 2019Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysiscited by 39
  2. 2022COGMEN: COntextualized GNN based Multimodal Emotion recognitioNcited by 29
  3. 2020Multilogue-Net: A Context Aware RNN for Multi-modal Emotion Detection and Sentiment Analysis in Conversationcited by 27
  4. 2023DeMT: Deformable Mixer Transformer for Multi-Task Learning of Dense Predictioncited by 27
  5. 2023Multimodal Multi-loss Fusion Network for Sentiment Analysiscited by 27
  6. 2023Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interactioncited by 18
  7. 2023Speech-Text Dialog Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignmentcited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Multimodal Adaptive Expert Selection with Text Routing and Ordinal Prototype Optimization for Sentiment Analysis
  2. 2026A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations
  3. 2025BTW: A Non-Parametric Variance Stabilization Framework for Multimodal Model Integration
  4. 2026Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?
  5. 2025Text-Routed Sparse Mixture-of-Experts Model with Explanation and Temporal Alignment for Multi-Modal Sentiment Analysis
  6. 2025Joint Learning using Mixture-of-Expert-Based Representation for Speech Enhancement and Robust Emotion Recognition
  7. 2026MVFA: A Multi-View Text-Guided Multimodal Fusion LLM Adapter for Sentiment Analysis and Emotion Recognition
  8. 2025Multi-Modality Collaborative Learning for Sentiment Analysis