# BLUE at Memotion 2.0 2022: You have my Image, my Text and my Transformer

arXiv: https://arxiv.org/abs/2202.07543 · HTML: https://mlsanity.com/paper/2202.07543/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 64 references that are in the map.
- First posted on arXiv 15 Feb 2022.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [Rank consistent ordinal regression for neural networks with application to age estimation](https://mlsanity.com/paper/1901.07884/) (2019)
- [Exploring Hate Speech Detection in Multimodal Publications](https://mlsanity.com/paper/1910.03814/) (2019)
- [SemEval-2020 Task 8: Memotion Analysis -- The Visuo-Lingual Metaphor!](https://mlsanity.com/paper/2008.03781/) (2020)
- [SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification](https://mlsanity.com/paper/2004.14454/) (2020)
- [SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images](https://mlsanity.com/paper/2105.09284/) (2021)
- [Deep Learning Models for Multilingual Hate Speech Detection](https://mlsanity.com/paper/2004.06465/) (2020)
- [Humor Detection: A Transformer Gets the Last Laugh](https://mlsanity.com/paper/1909.00252/) (2019)
- [Detecting Sarcasm in Multimodal Social Platforms](https://mlsanity.com/paper/1608.02289/) (2016)
- [DeepHate: Hate Speech Detection via Multi-Faceted Text Representations](https://mlsanity.com/paper/2103.11799/) (2021)
- [CASCADE: Contextual Sarcasm Detection in Online Discussion Forums](https://mlsanity.com/paper/1805.06413/) (2018)
- [Early Risk Detection of Pathological Gambling, Self-Harm and Depression Using BERT](https://mlsanity.com/paper/2106.16175/) (2021)
- [IITK at SemEval-2020 Task 8: Unimodal and Bimodal Sentiment Analysis of Internet Memes](https://mlsanity.com/paper/2007.10822/) (2020)
- [MoNoise: Modeling Noise Using a Modular Normalization System](https://mlsanity.com/paper/1710.03476/) (2017)
- [Uncovering the Limits of Text-based Emotion Detection](https://mlsanity.com/paper/2109.01900/) (2021)
- [Sequence-to-Sequence Lexical Normalization with Multilingual Transformers](https://mlsanity.com/paper/2110.02869/) (2021)
- [Perceived and Intended Sarcasm Detection with Graph Attention Networks](https://mlsanity.com/paper/2110.04001/) (2021)

## Built on by (most cited first, 5 of 5)

- [Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities](https://mlsanity.com/paper/2406.07353/) (2024)
- [Memotion 3: Dataset on Sentiment and Emotion Analysis of Codemixed Hindi-English Memes](https://mlsanity.com/paper/2303.09892/) (2023)
- [NYCU-TWO at Memotion 3: Good Foundation, Good Teacher, then you have Good Meme Analysis](https://mlsanity.com/paper/2302.06078/) (2023)
- [Meme Sentiment Analysis Enhanced with Multimodal Spatial Encoding and Facial Embedding](https://mlsanity.com/paper/2303.01781/) (2023)
- [Unimodal Intermediate Training for Multimodal Meme Sentiment Classification](https://mlsanity.com/paper/2308.00528/) (2023)

## Abstract (arXiv)

Memes are prevalent on the internet and continue to grow and evolve alongside our culture. An automatic understanding of memes propagating on the internet can shed light on the general sentiment and cultural attitudes of people. In this work, we present team BLUE's solution for the second edition of the MEMOTION shared task. We showcase two approaches for meme classification (i.e. sentiment, humour, offensive, sarcasm and motivation levels) using a text-only method using BERT, and a Multi-Modal-Multi-Task transformer network that operates on both the meme image and its caption to output the final scores. In both approaches, we leverage state-of-the-art pretrained models for text (BERT, Sentence Transformer) and image processing (EfficientNetV4, CLIP). Through our efforts, we obtain first place in task A, second place in task B and third place in task C. In addition, our team obtained the highest average score for all three tasks.