# Multi-Task Bidirectional Transformer Representations for Irony Detection

arXiv: https://arxiv.org/abs/1909.03526 · HTML: https://mlsanity.com/paper/1909.03526/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Sep 2019.

## Built on by (most cited first, 2 of 2)

- [Let-Mi: An Arabic Levantine Twitter Dataset for Misogynistic Language](https://mlsanity.com/paper/2103.10195/) (2021)
- [Zero-Resource Multi-Dialectal Arabic Natural Language Understanding](https://mlsanity.com/paper/2104.06591/) (2021)

## Abstract (arXiv)

Supervised deep learning requires large amounts of training data. In the context of the FIRE2019 Arabic irony detection shared task (IDAT@FIRE2019), we show how we mitigate this need by fine-tuning the pre-trained bidirectional encoders from transformers (BERT) on gold data in a multi-task setting. We further improve our models by by further pre-training BERT on `in-domain' data, thus alleviating an issue of dialect mismatch in the Google-released BERT model. Our best model acquires 82.4 macro F1 score, and has the unique advantage of being feature-engineering free (i.e., based exclusively on deep learning).