mlsanityData to 7 Oct 2026

Hybrid Multimodal Fusion for Humor Detection

arXiv 2209.11949 ↗First posted 24 Sep 2022cs.LG cs.AI cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we present our solution to the MuSe-Humor sub-challenge of the Multimodal Emotional Challenge (MuSe) 2022. The goal of the MuSe-Humor sub-challenge is to detect humor and calculate AUC from audiovisual recordings of German football Bundesliga press conferences. It is annotated for humor displayed by the coaches. For this sub-challenge, we first build a discriminant model using the transformer module and BiLSTM module, and then propose a hybrid fusion strategy to use the prediction results of each modality to improve the performance of the model. Our experiments demonstrate the effectiveness of our proposed model and hybrid fusion strategy on multimodal fusion, and the AUC of our proposed model on the test set is 0.8972.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 20 of its 46 references that are in the map. First posted on arXiv 24 Sep 2022.

Builds on

20 of 46 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
  6. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
  7. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  8. 2015WIDER FACE: A Face Detection Benchmarkcited by 297
  9. 2018Efficient Low-rank Multimodal Fusion with Modality-Specific Factorscited by 268
  10. 2016MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videoscited by 193
  11. 2018Memory Fusion Network for Multi-view Sequential Learningcited by 167
  12. 2019UR-FUNNY: A Multimodal Language Dataset for Understanding Humorcited by 130
8 more
  1. 2018Multi-attention Recurrent Network for Human Communication Comprehensioncited by 89
  2. 2018Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviorscited by 89
  3. 2018Multimodal Language Analysis with Recurrent Multistage Fusioncited by 43
  4. 2016Select-Additive Learning: Improving Generalization in Multimodal Sentiment Analysiscited by 29
  5. 2022The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stresscited by 23
  6. 2018Investigating Audio, Visual, and Text Fusion Methods for End-to-End Automatic Personality Predictioncited by 22
  7. 2018Modality-based Factorization for Multimodal Fusioncited by 7
  8. 2021Hybrid Mutimodal Fusion for Dimensional Emotion Recognitioncited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 4; 2025: 1.

9 papers.

  1. 2023Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generationcited by 56
  2. 2023The MuSe 2023 Multimodal Sentiment Analysis Challenge: Mimicked Emotions, Cross-Cultural Humour, and Personalisationcited by 12
  3. 2024Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leapscited by 12
  4. 2025A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Modelscited by 10
  5. 2024Systematic Literature Review: Computational Approaches for Humour Style Classificationcited by 6
  6. 2024Efficient Feature Extraction and Late Fusion Strategy for Audiovisual Emotional Mimicry Intensity Estimationcited by 6
  7. 2022Towards Multimodal Prediction of Spontaneous Humour: A Novel Dataset and First Resultscited by 4
  8. 2024The MuSe 2024 Multimodal Sentiment Analysis Challenge: Social Perception and Humor Recognitioncited by 3
  9. 2023From Generalized Laughter to Personalized Chuckles: Unleashing the Power of Data Fusion in Subjective Humor Detectioncited by 0