mlsanityData to 7 Oct 2026

Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)

arXiv 1906.01815 ↗First posted 5 Jun 2019cs.CL cs.CVcited by 173

Not on the map. Why?

Abstract (arXiv)

Sarcasm is often expressed through several verbal and non-verbal cues, e.g., a change of tone, overemphasis in a word, a drawn-out syllable, or a straight looking face. Most of the recent work in sarcasm detection has been carried out on textual data. In this paper, we argue that incorporating multimodal cues can improve the automatic classification of sarcasm. As a first step towards enabling the development of multimodal approaches for sarcasm detection, we propose a new sarcasm dataset, Multimodal Sarcasm Detection Dataset (MUStARD), compiled from popular TV shows. MUStARD consists of audiovisual utterances annotated with sarcasm labels. Each utterance is accompanied by its context of historical utterances in the dialogue, which provides additional information on the scenario where the utterance occurs. Our initial results show that the use of multimodal information can reduce the relative error rate of sarcasm detection by up to 12.9% in F-score when compared to the use of individual modalities. The full dataset is publicly available for use at https://github.com/soujanyaporia/MUStARD

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 19; 2023: 24; 2024: 30; 2025: 43; 2026: 44.

173 papers.

Sort:

By area:

  1. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 281
  2. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  3. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  4. 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
  5. 2023Quantifying & Modeling Multimodal Interactions: An Information Decomposition Frameworkcited by 81
  6. 2021Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systemscited by 73
  7. 2023Factorized Contrastive Learning: Going Beyond Multi-view Redundancycited by 73
  8. 2024Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversationscited by 45
  9. 2022On the Utility of Self-supervised Models for Prosody-related Taskscited by 42
  10. 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
  11. 2024What to align in multimodal contrastive learning?cited by 41
  12. 2021Multimodal Learning using Optimal Transport for Sarcasm and Humor Detectioncited by 36
38 more
  1. 2022Modular and Parameter-Efficient Multimodal Fusion with Promptingcited by 35
  2. 2023FunQA: Towards Surprising Video Comprehensioncited by 35
  3. 2024Distilling an End-to-End Voice Assistant Without Instruction Training Datacited by 35
  4. 2021Multi-modal Sarcasm Detection and Humor Classification in Code-mixed Conversationscited by 33
  5. 2022When did you become so smart, oh wise one?! Sarcasm Explanation in Multi-modal Multi-party Dialoguescited by 32
  6. 2025EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Modelscited by 32
  7. 2023SpeechPrompt v2: Prompt Tuning for Speech Classification Taskscited by 31
  8. 2022A Multimodal Corpus for Emotion Recognition in Sarcasmcited by 29
  9. 2023SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Modelscited by 28
  10. 2025Understanding the Emergence of Multimodal Representation Alignmentcited by 28
  11. 2021Nice perfume. How long did you marinate in it? Multimodal Sarcasm Explanationcited by 25
  12. 2023Paralinguistics-Enhanced Large Language Modeling of Spoken Dialoguecited by 25
  13. 2023MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Expertscited by 21
  14. 2023Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applicationscited by 20
  15. 2024A Survey of Multimodal Sarcasm Detectioncited by 19
  16. 2021See, Hear, Read: Leveraging Multimodality with Guided Attention for Abstractive Text Summarizationcited by 18
  17. 2024Neuro-Inspired Information-Theoretic Hierarchical Perception for Multimodal Learningcited by 18
  18. 2024Can Large Multimodal Models Uncover Deep Semantics Behind Images?cited by 16
  19. 2024Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Surveycited by 16
  20. 2024Generative Expressive Conversational Speech Synthesiscited by 16
  21. 2024XMeCap: Meme Caption Generation with Sub-Image Adaptabilitycited by 14
  22. 2025To Align or Not to Align: Strategic Multimodal Representation Alignment for Optimal Performancecited by 12
  23. 2023Can Language Models Laugh at YouTube Short-form Videos?cited by 11
  24. 2023Multimodal Fusion Interactions: A Study of Human and Automatic Quantificationcited by 9
  25. 2023UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructionscited by 9
  26. 2024FunnyNet-W: Multimodal Learning of Funny Moments in Videos in the Wildcited by 9
  27. 2024YesBut: A High-Quality Annotated Multimodal Dataset for evaluating Satire Comprehension capability of Vision-Language Modelscited by 9
  28. 2025Large Language Models for Subjective Language Understanding: A Surveycited by 9
  29. 2021!Qu'e maravilla! Multimodal Sarcasm Detection in Spanish: a Dataset and a Baselinecited by 8
  30. 2021Learning De-identified Representations of Prosody from Raw Audiocited by 8
  31. 2021TEASEL: A Transformer-Based Speech-Prefixed Language Modelcited by 8
  32. 2024BLSP-KD: Bootstrapping Language-Speech Pre-training via Knowledge Distillationcited by 8
  33. 2025Mind the Gap! Static and Interactive Evaluations of Large Audio Modelscited by 8
  34. 2025Can Large Language Models Help Multimodal Language Analysis? MMLA: A Comprehensive Benchmarkcited by 8
  35. 2026Unveiling the Cognitive Compass: Theory-of-Mind-Guided Multimodal Emotion Reasoningcited by 8
  36. 2023Sarcasm in Sight and Sound: Benchmarking and Expansion to Improve Multimodal Sarcasm Detectioncited by 7
  37. 2023On Sarcasm Detection with OpenAI GPT-based Modelscited by 7
  38. 2023DocMSU: A Comprehensive Benchmark for Document-level Multimodal Sarcasm Understandingcited by 7

50 most cited of 173 All 173 (JSON)