mlsanityData to 7 Oct 2026

The MuSe 2023 Multimodal Sentiment Analysis Challenge: Mimicked Emotions, Cross-Cultural Humour, and Personalisation

arXiv 2305.03369 ↗First posted 5 May 2023cs.LG cs.AI cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

The MuSe 2023 is a set of shared tasks addressing three different contemporary multimodal affect and sentiment analysis problems: In the Mimicked Emotions Sub-Challenge (MuSe-Mimic), participants predict three continuous emotion targets. This sub-challenge utilises the Hume-Vidmimic dataset comprising of user-generated videos. For the Cross-Cultural Humour Detection Sub-Challenge (MuSe-Humour), an extension of the Passau Spontaneous Football Coach Humour (Passau-SFCH) dataset is provided. Participants predict the presence of spontaneous humour in a cross-cultural setting. The Personalisation Sub-Challenge (MuSe-Personalisation) is based on the Ulm-Trier Social Stress Test (Ulm-TSST) dataset, featuring recordings of subjects in a stressed situation. Here, arousal and valence signals are to be predicted, whereas parts of the test labels are made available in order to facilitate personalisation. MuSe 2023 seeks to bring together a broad audience from different research communities such as audio-visual emotion recognition, natural language processing, signal processing, and health informatics. In this baseline paper, we introduce the datasets, sub-challenges, and provided feature sets. As a competitive baseline system, a Gated Recurrent Unit (GRU)-Recurrent Neural Network (RNN) is employed. On the respective sub-challenges' test datasets, it achieves a mean (across three continuous intensity targets) Pearson's Correlation Coefficient of .4727 for MuSe-Mimic, an Area Under the Curve (AUC) value of .8310 for MuSe-Humor and Concordance Correlation Coefficient (CCC) values of .7482 for arousal and .7827 for valence in the MuSe-Personalisation sub-challenge.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 22 of its 71 references that are in the map. First posted on arXiv 5 May 2023.

Builds on

22 of 71 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  5. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
  6. 2019How multilingual is Multilingual BERT?cited by 642
  7. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 158
  8. 2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
  9. 2019UR-FUNNY: A Multimodal Language Dataset for Understanding Humorcited by 130
  10. 2022Audio Self-supervised Learning: A Surveycited by 65
  11. 2022Speech Emotion Recognition using Self-Supervised Featurescited by 52
  12. 2020PySBD: Pragmatic Sentence Boundary Disambiguationcited by 38
10 more
  1. 2021The INTERSPEECH 2021 Computational Paralinguistics Challenge: COVID-19 Cough, COVID-19 Speech, Escalation & Primatescited by 32
  2. 2021The MuSe 2021 Multimodal Sentiment Analysis Challenge: Sentiment, Emotion, Physiological-Emotion, and Stresscited by 27
  3. 2022The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stresscited by 23
  4. 2021EmoNet: A Transfer Learning Framework for Multi-Corpus Speech Emotion Recognitioncited by 17
  5. 2021DeepSpectrumLite: A Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing from Decentralised Datacited by 12
  6. 2022Hybrid Multimodal Feature Extraction, Mining and Fusion for Sentiment Analysiscited by 12
  7. 2021"So You Think You're Funny?": Rating the Humour Quotient in Standup Comedycited by 11
  8. 2022Hybrid Multimodal Fusion for Humor Detectioncited by 9
  9. 2021MuSe-Toolbox: The Multimodal Sentiment Analysis Continuous Annotation Fusion and Discrete Class Transformation Toolboxcited by 8
  10. 2023Beyond One-Size-Fits-All: A Survey of Personalized Affective Computing in Human-Agent Interactioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 1; 2026: 2.

12 papers.

  1. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  2. 2024Can Large Multimodal Models Uncover Deep Semantics Behind Images?cited by 16
  3. 2024Unimodal Multi-Task Fusion for Emotional Mimicry Intensity Predictioncited by 6
  4. 2025Semantic Matters: Multimodal Features for Affective Analysiscited by 5
  5. 2024Modeling Emotional Trajectories in Written Stories Utilizing Transformers and Weakly-Supervised Learningcited by 4
  6. 2024The MuSe 2024 Multimodal Sentiment Analysis Challenge: Social Perception and Humor Recognitioncited by 3
  7. 2026The 2026 ACII Dyadic Conversations (DaiKon) Workshop & Challengecited by 1
  8. 2023Exploiting Diverse Feature for Multimodal Sentiment Analysiscited by 0
  9. 2023From Generalized Laughter to Personalized Chuckles: Unleashing the Power of Data Fusion in Subjective Humor Detectioncited by 0
  10. 2024Improving Personalisation in Valence and Arousal Prediction using Data Augmentationcited by 0
  11. 2024This Paper Had the Smartest Reviewers -- Flattery Detection Utilising an Audio-Textual Transformer-Based Approachcited by 0
  12. 2026Evaluating multimodal emotion recognition in proactive conversational agents: A user studycited by 0