# When did you become so smart, oh wise one?! Sarcasm Explanation in Multi-modal Multi-party Dialogues

arXiv: https://arxiv.org/abs/2203.06419 · HTML: https://mlsanity.com/paper/2203.06419/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 12 Mar 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)](https://mlsanity.com/paper/1906.01815/) (2019)
- [Reasoning with Sarcasm by Reading In-between](https://mlsanity.com/paper/1805.02856/) (2018)
- [Multi-modal Sarcasm Detection and Humor Classification in Code-mixed Conversations](https://mlsanity.com/paper/2105.09984/) (2021)
- [R3: Reverse, Retrieve, and Rank for Sarcasm Generation with Commonsense Knowledge](https://mlsanity.com/paper/2004.13248/) (2020)
- [Context-Aware Self-Attention Networks](https://mlsanity.com/paper/1902.05766/) (2019)
- [Automatic Sarcasm Detection: A Survey](https://mlsanity.com/paper/1602.03426/) (2016)
- [Sarcasm SIGN: Interpreting Sarcasm with Sentiment Based Monolingual Machine Translation](https://mlsanity.com/paper/1704.06836/) (2017)
- [Sarcasm Analysis using Conversation Context](https://mlsanity.com/paper/1808.07531/) (2018)
- [The Role of Conversation Context for Sarcasm Detection in Online Interactions](https://mlsanity.com/paper/1707.06226/) (2017)
- [A Corpus of English-Hindi Code-Mixed Tweets for Sarcasm Detection](https://mlsanity.com/paper/1805.11869/) (2018)
- [Are you serious?: Rhetorical Questions and Sarcasm in Social Media Dialog](https://mlsanity.com/paper/1709.05305/) (2017)

## Built on by (most cited first, 30 of 33)

- [FLUTE: Figurative Language Understanding through Textual Explanations](https://mlsanity.com/paper/2205.12404/) (2022)
- [Multi-source Semantic Graph-based Multimodal Sarcasm Explanation Generation](https://mlsanity.com/paper/2306.16650/) (2023)
- [Can Language Models Laugh at YouTube Short-form Videos?](https://mlsanity.com/paper/2310.14159/) (2023)
- [Tox-BART: Leveraging Toxicity Attributes for Explanation Generation of Implicit Hate Speech](https://mlsanity.com/paper/2406.03953/) (2024)
- [Experience and Evidence are the eyes of an excellent summarizer! Towards Knowledge Infused Multi-modal Clinical Conversation Summarization](https://mlsanity.com/paper/2309.15739/) (2023)
- [A Survey in Automatic Irony Processing: Linguistic, Cognitive, and Multi-X Perspectives](https://mlsanity.com/paper/2209.04712/) (2022)
- [Explaining (Sarcastic) Utterances to Enhance Affect Understanding in Multimodal Dialogues](https://mlsanity.com/paper/2211.11049/) (2022)
- [Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue](https://mlsanity.com/paper/2402.03658/) (2024)
- [SemEval 2024 -- Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF)](https://mlsanity.com/paper/2402.18944/) (2024)
- [VAGUE: Visual Contexts Clarify Ambiguous Expressions](https://mlsanity.com/paper/2411.14137/) (2024)
- [PunchBench: Benchmarking MLLMs in Multimodal Punchline Comprehension](https://mlsanity.com/paper/2412.11906/) (2024)
- [Seeing Sarcasm Through Different Eyes: Analyzing Multimodal Sarcasm Perception in Large Vision-Language Models](https://mlsanity.com/paper/2503.12149/) (2025)
- [Target-Augmented Shared Fusion-based Multimodal Sarcasm Explanation Generation](https://mlsanity.com/paper/2502.07391/) (2025)
- [IRONIC: Coherence-Aware Reasoning Chains for Multi-Modal Sarcasm Detection](https://mlsanity.com/paper/2505.16258/) (2025)
- [Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding](https://mlsanity.com/paper/2508.20765/) (2025)
- [From Multilingual Complexity to Emotional Clarity: Leveraging Commonsense to Unveil Emotions in Code-Mixed Dialogues](https://mlsanity.com/paper/2310.13080/) (2023)
- [Exploring Multilingual Unseen Speaker Emotion Recognition: Leveraging Co-Attention Cues in Multitask Learning](https://mlsanity.com/paper/2406.08931/) (2024)
- [Unlocking Financial Insights: An advanced Multimodal Summarization with Multimodal Output Framework for Financial Advisory Videos](https://mlsanity.com/paper/2509.20961/) (2025)
- [Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems](https://mlsanity.com/paper/2307.07255/) (2023)
- [Interaction is all You Need? A Study of Robots Ability to Understand and Execute](https://mlsanity.com/paper/2311.07150/) (2023)
- [Harmonizing Code-mixed Conversations: Personality-assisted Code-mixed Response Generation in Dialogues](https://mlsanity.com/paper/2401.12995/) (2024)
- [Think out Loud: Emotion Deducing Explanation in Dialogues](https://mlsanity.com/paper/2406.04758/) (2024)
- [v-HUB: A Benchmark for Video Humor Understanding from Vision and Sound](https://mlsanity.com/paper/2509.25773/) (2025)
- [Yes, this is what I was looking for! Towards Multi-modal Medical Consultation Concern Summary Generation](https://mlsanity.com/paper/2401.05134/) (2024)
- [Retrieval Augmented Cross-Modal Tag Recommendation in Software Q&A Sites](https://mlsanity.com/paper/2402.03635/) (2024)
- [MasonTigers at SemEval-2024 Task 10: Emotion Discovery and Flip Reasoning in Conversation with Ensemble of Transformers and Prompting](https://mlsanity.com/paper/2407.00581/) (2024)
- [Generative Emotion Cause Explanation in Multimodal Conversations](https://mlsanity.com/paper/2411.02430/) (2024)
- [Was that Sarcasm?: A Literature Survey on Sarcasm Detection](https://mlsanity.com/paper/2412.00425/) (2024)
- [Nek Minit: Harnessing Pragmatic Metacognitive Prompting for Explainable Sarcasm Detection of Australian and Indian English](https://mlsanity.com/paper/2505.15095/) (2025)
- [Multimodal Fine-grained Reasoning for Post Quality Evaluation](https://mlsanity.com/paper/2507.17934/) (2025)

## Abstract (arXiv)

Indirect speech such as sarcasm achieves a constellation of discourse goals in human communication. While the indirectness of figurative language warrants speakers to achieve certain pragmatic goals, it is challenging for AI agents to comprehend such idiosyncrasies of human communication. Though sarcasm identification has been a well-explored topic in dialogue analysis, for conversational systems to truly grasp a conversation's innate meaning and generate appropriate responses, simply detecting sarcasm is not enough; it is vital to explain its underlying sarcastic connotation to capture its true essence. In this work, we study the discourse structure of sarcastic conversations and propose a novel task - Sarcasm Explanation in Dialogue (SED). Set in a multimodal and code-mixed setting, the task aims to generate natural language explanations of satirical conversations. To this end, we curate WITS, a new dataset to support our task. We propose MAF (Modality Aware Fusion), a multimodal context-aware attention and global information fusion module to capture multimodality and use it to benchmark WITS. The proposed attention module surpasses the traditional multimodal fusion baselines and reports the best performance on almost all metrics. Lastly, we carry out detailed analyses both quantitatively and qualitatively.