mlsanityData to 14 Oct 2026

Leveraging Label Potential for Enhanced Multimodal Emotion Recognition

arXiv 2504.05158 ↗First posted 7 Apr 2025cs.SD cs.AI eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal emotion recognition (MER) seeks to integrate various modalities to predict emotional states accurately. However, most current research focuses solely on the fusion of audio and text features, overlooking the valuable information in emotion labels. This oversight could potentially hinder the performance of existing methods, as emotion labels harbor rich, insightful information that could significantly aid MER. We introduce a novel model called Label Signal-Guided Multimodal Emotion Recognition (LSGMER) to overcome this limitation. This model aims to fully harness the power of emotion label information to boost the classification accuracy and stability of MER. Specifically, LSGMER employs a Label Signal Enhancement module that optimizes the representation of modality features by interacting with audio and text features through label embeddings, enabling it to capture the nuances of emotions precisely. Furthermore, we propose a Joint Objective Optimization(JOO) approach to enhance classification accuracy by introducing the Attribution-Prediction Consistency Constraint (APC), which strengthens the alignment between fused features and emotion categories. Extensive experiments conducted on the IEMOCAP and MELD datasets have demonstrated the effectiveness of our proposed LSGMER model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 58 of its 70 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

58 of 70 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  8. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  9. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  10. 2020Adaptive Federated Optimizationcited by 1,046
  11. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  12. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
38 more
  1. 2019Towards Federated Learning at Scale: System Designcited by 832
  2. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  3. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
  4. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 543
  5. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  6. 2019The State of Sparsity in Deep Neural Networkscited by 476
  7. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  8. 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 328
  9. 2017Implicit Regularization in Matrix Factorizationcited by 322
  10. 2021A Field Guide to Federated Optimizationcited by 315
  11. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
  12. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  13. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
  14. 2019Sparse Networks from Scratch: Faster Training without Losing Performancecited by 213
  15. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 204
  16. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
  17. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  18. 2019Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterizationcited by 189
  19. 2023Understanding plasticity in neural networkscited by 174
  20. 2015Conditional Computation in Neural Networks for faster modelscited by 167
  21. 2020Soft Threshold Weight Reparameterization for Learnable Sparsitycited by 151
  22. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 125
  23. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 118
  24. 2017Bayesian Compression for Deep Learningcited by 116
  25. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 90
  26. 2022Federated Learning Enables Big Data for Rare Cancer Boundary Detectioncited by 84
  27. 2019Implicit Regularization for Optimal Sparse Recoverycited by 80
  28. 2021On Large-Cohort Training for Federated Learningcited by 74
  29. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  30. 2020Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layerscited by 68
  31. 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
  32. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 55
  33. 2021Powerpropagation: A sparsity inducing weight reparameterisationcited by 49
  34. 2020Sparse Weight Activation Trainingcited by 45
  35. 2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45
  36. 2016Generalization Error of Invariant Classifierscited by 42
  37. 2019Semi-Cyclic Stochastic Gradient Descentcited by 40
  38. 2022Achieving Personalized Federated Learning with Sparse Local Modelscited by 31

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.