# Multimodal Emotion Recognition and Sentiment Analysis in Multi-Party Conversation Contexts

arXiv: https://arxiv.org/abs/2503.06805 · HTML: https://mlsanity.com/paper/2503.06805/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 21 references that are in the map.
- First posted on arXiv 9 Mar 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [M2FNet: Multi-modal Fusion Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/2206.02187/) (2022)
- [Beyond Silent Letters: Amplifying LLMs in Emotion Recognition with Vocal Nuances](https://mlsanity.com/paper/2407.21315/) (2024)
- [Supervised Adversarial Contrastive Learning for Emotion Recognition in Conversations](https://mlsanity.com/paper/2306.01505/) (2023)
- [TelME: Teacher-leading Multimodal Fusion Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/2401.12987/) (2024)
- [Continuous Sign Language Recognition Using Intra-inter Gloss Attention](https://mlsanity.com/paper/2406.18333/) (2024)
- [Facial Emotion Recognition Under Mask Coverage Using a Data Augmentation Technique](https://mlsanity.com/paper/2312.01335/) (2023)
- [Comparative Analysis of Modality Fusion Approaches for Audio-Visual Person Identification and Verification](https://mlsanity.com/paper/2409.00562/) (2024)

## Built on by (most cited first, 2 of 2)

- [EmoScene: A Dual-space Dataset for Controllable Affective Image Generation](https://mlsanity.com/paper/2604.00933/) (2026)
- [AMB-DSGDN: Adaptive Modality-Balanced Dynamic Semantic Graph Differential Network for Multimodal Emotion Recognition](https://mlsanity.com/paper/2603.10043/) (2026)

## Abstract (arXiv)

Emotion recognition and sentiment analysis are pivotal tasks in speech and language processing, particularly in real-world scenarios involving multi-party, conversational data. This paper presents a multimodal approach to tackle these challenges on a well-known dataset. We propose a system that integrates four key modalities/channels using pre-trained models: RoBERTa for text, Wav2Vec2 for speech, a proposed FacialNet for facial expressions, and a CNN+Transformer architecture trained from scratch for video analysis. Feature embeddings from each modality are concatenated to form a multimodal vector, which is then used to predict emotion and sentiment labels. The multimodal system demonstrates superior performance compared to unimodal approaches, achieving an accuracy of 66.36% for emotion recognition and 72.15% for sentiment analysis.