mlsanityData to 14 Oct 2026

UR-FUNNY: A Multimodal Language Dataset for Understanding Humor

arXiv 1904.06618 ↗First posted 14 Apr 2019cs.LG cs.CL stat.MLcited by 132

Not on the map. Why?

Abstract (arXiv)

Humor is a unique and creative communicative behavior displayed during social interactions. It is produced in a multimodal manner, through the usage of words (text), gestures (vision) and prosodic cues (acoustic). Understanding humor from these three modalities falls within boundaries of multimodal language; a recent research trend in natural language processing that models natural language as it happens in face-to-face communication. Although humor detection is an established research area in NLP, in a multimodal context it is an understudied area. This paper presents a diverse multimodal dataset, called UR-FUNNY, to open the door to understanding multimodal language used in expressing humor. The dataset and accompanying studies, present a framework in multimodal humor detection for the natural language processing community. UR-FUNNY is publicly available for research.

Key facts. Cited by 132 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 15; 2023: 18; 2024: 26; 2025: 30; 2026: 32.

132 papers.

Sort:

By area:

  1. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 146
  2. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  3. 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
  4. 2023Quantifying & Modeling Multimodal Interactions: An Information Decomposition Frameworkcited by 83
  5. 2023Factorized Contrastive Learning: Going Beyond Multi-view Redundancycited by 75
  6. 2023Boosting Multi-modal Model Performance with Adaptive Gradient Modulationcited by 73
  7. 2023SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalizationcited by 48
  8. 2024What to align in multimodal contrastive learning?cited by 47
  9. 2021Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysiscited by 45
  10. 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
  11. 2024Distilling an End-to-End Voice Assistant Without Instruction Training Datacited by 39
  12. 2021Multimodal Learning using Optimal Transport for Sarcasm and Humor Detectioncited by 37
38 more
  1. 2023FunQA: Towards Surprising Video Comprehensioncited by 35
  2. 2021Multi-modal Sarcasm Detection and Humor Classification in Code-mixed Conversationscited by 33
  3. 2025EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Modelscited by 32
  4. 2022High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learningcited by 31
  5. 2025Understanding the Emergence of Multimodal Representation Alignmentcited by 31
  6. 2023A Low-rank Matching Attention based Cross-modal Feature Fusion Method for Conversational Emotion Recognitioncited by 29
  7. 2023SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Modelscited by 28
  8. 2022i-Code: An Integrative and Composable Multimodal Learning Frameworkcited by 24
  9. 2022The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stresscited by 23
  10. 2021The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvementscited by 22
  11. 2022ExPUNations: Augmenting Puns with Keywords and Explanationscited by 21
  12. 2023Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applicationscited by 21
  13. 2023MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Expertscited by 21
  14. 2024Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Surveycited by 16
  15. 2024Is Sarcasm Detection A Step-by-Step Reasoning Process in Large Language Models?cited by 14
  16. 2024XMeCap: Meme Caption Generation with Sub-Image Adaptabilitycited by 14
  17. 2024Can Pre-trained Language Models Understand Chinese Humor?cited by 13
  18. 2023The MuSe 2023 Multimodal Sentiment Analysis Challenge: Mimicked Emotions, Cross-Cultural Humour, and Personalisationcited by 12
  19. 2021"So You Think You're Funny?": Rating the Humour Quotient in Standup Comedycited by 11
  20. 2023Can Language Models Laugh at YouTube Short-form Videos?cited by 11
  21. 2024Debiased Multimodal Understanding for Human Language Sequencescited by 10
  22. 2022Hybrid Multimodal Fusion for Humor Detectioncited by 9
  23. 2023Multimodal Fusion Interactions: A Study of Human and Automatic Quantificationcited by 9
  24. 2024FunnyNet-W: Multimodal Learning of Funny Moments in Videos in the Wildcited by 9
  25. 2024YesBut: A High-Quality Annotated Multimodal Dataset for evaluating Satire Comprehension capability of Vision-Language Modelscited by 9
  26. 2025Can Large Language Models Help Multimodal Language Analysis? MMLA: A Comprehensive Benchmarkcited by 9
  27. 2025StandUp4AI: A New Multilingual Dataset for Humor Detection in Stand-up Comedy Videoscited by 9
  28. 2021TEASEL: A Transformer-Based Speech-Prefixed Language Modelcited by 8
  29. 2025Mind the Gap! Static and Interactive Evaluations of Large Audio Modelscited by 8
  30. 2026Unveiling the Cognitive Compass: Theory-of-Mind-Guided Multimodal Emotion Reasoningcited by 8
  31. 2021M2H2: A Multimodal Multiparty Hindi Dataset For Humor Recognition in Conversationscited by 7
  32. 2024Creativity in AI: Progresses and Challengescited by 7
  33. 2025MAVERIX: Multimodal Audio-Visual Evaluation and Recognition IndeXcited by 7
  34. 2025InfMasking: Unleashing Synergistic Information by Contrastive Multimodal Interactionscited by 7
  35. 2025The More, the Merrier: Contrastive Fusion for Higher-Order Multimodal Alignmentcited by 7
  36. 2023TextMI: Textualize Multimodal Information for Integrating Non-verbal Cues in Pre-trained Language Modelscited by 6
  37. 2024Systematic Literature Review: Computational Approaches for Humour Style Classificationcited by 6
  38. 2024HumorDB: Can AI understand graphical humor?cited by 6

50 most cited of 132 All 132 (JSON)