# UPB at SemEval-2021 Task 7: Adversarial Multi-Task Learning for Detecting and Rating Humor and Offense

arXiv: https://arxiv.org/abs/2104.06063 · HTML: https://mlsanity.com/paper/2104.06063/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Adversarial Multi-task Learning for Text Classification](https://mlsanity.com/paper/1704.05742/) (2017)
- [Investigating Capsule Networks with Dynamic Routing for Text Classification](https://mlsanity.com/paper/1804.00538/) (2018)
- [UPB at SemEval-2020 Task 8: Joint Textual and Visual Modeling in a Multi-Task Learning Architecture for Memotion Analysis](https://mlsanity.com/paper/2009.02779/) (2020)
- [UPB at SemEval-2020 Task 12: Multilingual Offensive Language Detection on Social Media by Fine-tuning a Variety of BERT-based Models](https://mlsanity.com/paper/2010.13609/) (2020)
- [Towards Linear Time Neural Machine Translation with Capsule Networks](https://mlsanity.com/paper/1811.00287/) (2018)
- [UPB at SemEval-2020 Task 11: Propaganda Detection with Domain-Specific Trained BERT](https://mlsanity.com/paper/2009.05289/) (2020)
- [Event-Related Bias Removal for Real-time Disaster Events](https://mlsanity.com/paper/2011.00681/) (2020)
- [UPB at SemEval-2020 Task 9: Identifying Sentiment in Code-Mixed Social Media Texts using Transformers and Multi-Task Learning](https://mlsanity.com/paper/2009.02780/) (2020)
- [SRL4ORL: Improving Opinion Role Labeling using Multi-task Learning with Semantic Role Labeling](https://mlsanity.com/paper/1711.00768/) (2017)
- [Emotion Detection with Neural Personal Discrimination](https://mlsanity.com/paper/1908.10703/) (2019)

## Abstract (arXiv)

Detecting humor is a challenging task since words might share multiple valences and, depending on the context, the same words can be even used in offensive expressions. Neural network architectures based on Transformer obtain state-of-the-art results on several Natural Language Processing tasks, especially text classification. Adversarial learning, combined with other techniques such as multi-task learning, aids neural models learn the intrinsic properties of data. In this work, we describe our adversarial multi-task network, AMTL-Humor, used to detect and rate humor and offensive texts from Task 7 at SemEval-2021. Each branch from the model is focused on solving a related task, and consists of a BiLSTM layer followed by Capsule layers, on top of BERTweet used for generating contextualized embeddings. Our best model consists of an ensemble of all tested configurations, and achieves a 95.66% F1-score and 94.70% accuracy for Task 1a, while obtaining RMSE scores of 0.6200 and 0.5318 for Tasks 1b and 2, respectively.