mlsanityData to 7 Oct 2026

Exploiting Diverse Feature for Multimodal Sentiment Analysis

arXiv 2308.13421 ↗First posted 25 Aug 2023cs.CV cs.MM cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we present our solution to the MuSe-Personalisation sub-challenge in the MuSe 2023 Multimodal Sentiment Analysis Challenge. The task of MuSe-Personalisation aims to predict the continuous arousal and valence values of a participant based on their audio-visual, language, and physiological signal modalities data. Considering different people have personal characteristics, the main challenge of this task is how to build robustness feature presentation for sentiment prediction. To address this issue, we propose exploiting diverse features. Specifically, we proposed a series of feature extraction methods to build a robust representation and model ensemble. We empirically evaluate the performance of the utilized method on the officially provided dataset. As a result, we achieved 3rd place in the MuSe-Personalisation sub-challenge. Specifically, we achieve the results of 0.8492 and 0.8439 for MuSe-Personalisation in terms of arousal and valence CCC.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 29 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

16 of 29 references.

References · 16

  1. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  5. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  6. 2021TransFER: Learning Relation-aware Facial Expression Representations with Transformerscited by 85
  7. 2016AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challengecited by 68
  8. 2022Vision Transformer with Attentive Pooling for Robust Facial Expression Recognitioncited by 35
  9. 2021The MuSe 2021 Multimodal Sentiment Analysis Challenge: Sentiment, Emotion, Physiological-Emotion, and Stresscited by 27
  10. 2023Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classificationcited by 24
  11. 2022The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stresscited by 23
  12. 2023ViGT: Proposal-free Video Grounding with Learnable Token in Transformercited by 22
4 more
  1. 2023Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videoscited by 15
  2. 2022Hybrid Multimodal Feature Extraction, Mining and Fusion for Sentiment Analysiscited by 12
  3. 2023The MuSe 2023 Multimodal Sentiment Analysis Challenge: Mimicked Emotions, Cross-Cultural Humour, and Personalisationcited by 12
  4. 2023Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformerscited by 5

Built on by

Not cited yet.