# Interpretable Multi-Head Self-Attention model for Sarcasm Detection in social media

arXiv: https://arxiv.org/abs/2101.05875 · HTML: https://mlsanity.com/paper/2101.05875/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 14 Jan 2021.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [A Large Self-Annotated Corpus for Sarcasm](https://mlsanity.com/paper/1704.05579/) (2017)
- [Creating and Characterizing a Diverse Corpus of Sarcasm in Dialogue](https://mlsanity.com/paper/1709.05404/) (2017)
- [Reasoning with Sarcasm by Reading In-between](https://mlsanity.com/paper/1805.02856/) (2018)
- [A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks](https://mlsanity.com/paper/1610.08815/) (2016)
- [Modelling Context with User Embeddings for Sarcasm Detection in Social Media](https://mlsanity.com/paper/1607.00976/) (2016)
- [CASCADE: Contextual Sarcasm Detection in Online Discussion Forums](https://mlsanity.com/paper/1805.06413/) (2018)
- [Sarcasm Detection using Hybrid Neural Network](https://mlsanity.com/paper/1908.07414/) (2019)
- [Deep contextualized word representations for detecting sarcasm and irony](https://mlsanity.com/paper/1809.09795/) (2018)
- [Sarcasm Analysis using Conversation Context](https://mlsanity.com/paper/1808.07531/) (2018)
- [Forecasting the Success of Television Series using Machine Learning](https://mlsanity.com/paper/1910.12589/) (2019)
- [DeepFork: Supervised Prediction of Information Diffusion in GitHub](https://mlsanity.com/paper/1910.07999/) (2019)

## Abstract (arXiv)

Sarcasm is a linguistic expression often used to communicate the opposite of what is said, usually something that is very unpleasant with an intention to insult or ridicule. Inherent ambiguity in sarcastic expressions, make sarcasm detection very difficult. In this work, we focus on detecting sarcasm in textual conversations from various social networking platforms and online media. To this end, we develop an interpretable deep learning model using multi-head self-attention and gated recurrent units. Multi-head self-attention module aids in identifying crucial sarcastic cue-words from the input, and the recurrent units learn long-range dependencies between these cue-words to better classify the input text. We show the effectiveness of our approach by achieving state-of-the-art results on multiple datasets from social networking platforms and online media. Models trained using our proposed approach are easily interpretable and enable identifying sarcastic cues in the input text which contribute to the final classification score. We visualize the learned attention weights on few sample input texts to showcase the effectiveness and interpretability of our model.