# Explaining (Sarcastic) Utterances to Enhance Affect Understanding in Multimodal Dialogues

arXiv: https://arxiv.org/abs/2211.11049 · HTML: https://mlsanity.com/paper/2211.11049/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 20 Nov 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)](https://mlsanity.com/paper/1906.01815/) (2019)
- [Reasoning with Sarcasm by Reading In-between](https://mlsanity.com/paper/1805.02856/) (2018)
- [Multi-modal Sarcasm Detection and Humor Classification in Code-mixed Conversations](https://mlsanity.com/paper/2105.09984/) (2021)
- [When did you become so smart, oh wise one?! Sarcasm Explanation in Multi-modal Multi-party Dialogues](https://mlsanity.com/paper/2203.06419/) (2022)
- [R3: Reverse, Retrieve, and Rank for Sarcasm Generation with Commonsense Knowledge](https://mlsanity.com/paper/2004.13248/) (2020)
- [Context-Aware Self-Attention Networks](https://mlsanity.com/paper/1902.05766/) (2019)
- [Automatic Sarcasm Detection: A Survey](https://mlsanity.com/paper/1602.03426/) (2016)
- [Sarcasm SIGN: Interpreting Sarcasm with Sentiment Based Monolingual Machine Translation](https://mlsanity.com/paper/1704.06836/) (2017)
- [The Role of Conversation Context for Sarcasm Detection in Online Interactions](https://mlsanity.com/paper/1707.06226/) (2017)
- [A Corpus of English-Hindi Code-Mixed Tweets for Sarcasm Detection](https://mlsanity.com/paper/1805.11869/) (2018)

## Built on by (most cited first, 6 of 6)

- [Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue](https://mlsanity.com/paper/2402.03658/) (2024)
- [SemEval 2024 -- Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF)](https://mlsanity.com/paper/2402.18944/) (2024)
- [VAGUE: Visual Contexts Clarify Ambiguous Expressions](https://mlsanity.com/paper/2411.14137/) (2024)
- [Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems](https://mlsanity.com/paper/2307.07255/) (2023)
- [ViSP: A PPO-Driven Framework for Sarcasm Generation with Contrastive Learning](https://mlsanity.com/paper/2507.09482/) (2025)
- [MuVaC: A Variational Causal Framework for Multimodal Sarcasm Understanding in Dialogues](https://mlsanity.com/paper/2601.20451/) (2026)

## Abstract (arXiv)

Conversations emerge as the primary media for exchanging ideas and conceptions. From the listener's perspective, identifying various affective qualities, such as sarcasm, humour, and emotions, is paramount for comprehending the true connotation of the emitted utterance. However, one of the major hurdles faced in learning these affect dimensions is the presence of figurative language, viz. irony, metaphor, or sarcasm. We hypothesize that any detection system constituting the exhaustive and explicit presentation of the emitted utterance would improve the overall comprehension of the dialogue. To this end, we explore the task of Sarcasm Explanation in Dialogues, which aims to unfold the hidden irony behind sarcastic utterances. We propose MOSES, a deep neural network, which takes a multimodal (sarcastic) dialogue instance as an input and generates a natural language sentence as its explanation. Subsequently, we leverage the generated explanation for various natural language understanding tasks in a conversational dialogue setup, such as sarcasm detection, humour identification, and emotion recognition. Our evaluation shows that MOSES outperforms the state-of-the-art system for SED by an average of ~2% on different evaluation metrics, such as ROUGE, BLEU, and METEOR. Further, we observe that leveraging the generated explanation advances three downstream tasks for affect classification - an average improvement of ~14% F1-score in the sarcasm detection task and ~2% in the humour identification and emotion recognition task. We also perform extensive analyses to assess the quality of the results.