# Unimodal Intermediate Training for Multimodal Meme Sentiment Classification

arXiv: https://arxiv.org/abs/2308.00528 · HTML: https://mlsanity.com/paper/2308.00528/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 1 Aug 2023.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations](https://mlsanity.com/paper/1808.09121/) (2018)
- [Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks](https://mlsanity.com/paper/1811.01088/) (2018)
- [Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?](https://mlsanity.com/paper/2005.00628/) (2020)
- [MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets](https://mlsanity.com/paper/2109.05184/) (2021)
- [SemEval-2020 Task 8: Memotion Analysis -- The Visuo-Lingual Metaphor!](https://mlsanity.com/paper/2008.03781/) (2020)
- [DynaSent: A Dynamic Benchmark for Sentiment Analysis](https://mlsanity.com/paper/2012.15349/) (2020)
- [What to Pre-Train on? Efficient Intermediate Task Selection](https://mlsanity.com/paper/2104.08247/) (2021)
- [Transferability in Deep Learning: A Survey](https://mlsanity.com/paper/2201.05867/) (2022)
- [Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memes](https://mlsanity.com/paper/2012.07788/) (2020)
- [Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solution](https://mlsanity.com/paper/2012.08290/) (2020)
- [Large-Scale Hate Speech Detection with Cross-Domain Transfer](https://mlsanity.com/paper/2203.01111/) (2022)
- [Caption Enriched Samples for Improving Hateful Memes Detection](https://mlsanity.com/paper/2109.10649/) (2021)
- [AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Media](https://mlsanity.com/paper/2106.11229/) (2021)
- [IITK at SemEval-2020 Task 8: Unimodal and Bimodal Sentiment Analysis of Internet Memes](https://mlsanity.com/paper/2007.10822/) (2020)
- [Memotion 3: Dataset on Sentiment and Emotion Analysis of Codemixed Hindi-English Memes](https://mlsanity.com/paper/2303.09892/) (2023)
- [Exercise? I thought you said 'Extra Fries': Leveraging Sentence Demarcations and Multi-hop Attention for Meme Affect Analysis](https://mlsanity.com/paper/2103.12377/) (2021)
- [BLUE at Memotion 2.0 2022: You have my Image, my Text and my Transformer](https://mlsanity.com/paper/2202.07543/) (2022)
- [Domain-aware Self-supervised Pre-training for Label-Efficient Meme Analysis](https://mlsanity.com/paper/2209.14667/) (2022)
- [Meme Sentiment Analysis Enhanced with Multimodal Spatial Encoding and Facial Embedding](https://mlsanity.com/paper/2303.01781/) (2023)

## Built on by (most cited first, 1 of 1)

- [What Makes a Meme a Meme? Identifying Memes for Memetics-Aware Dataset Creation](https://mlsanity.com/paper/2407.11861/) (2024)

## Abstract (arXiv)

Internet Memes remain a challenging form of user-generated content for automated sentiment classification. The availability of labelled memes is a barrier to developing sentiment classifiers of multimodal memes. To address the shortage of labelled memes, we propose to supplement the training of a multimodal meme classifier with unimodal (image-only and text-only) data. In this work, we present a novel variant of supervised intermediate training that uses relatively abundant sentiment-labelled unimodal data. Our results show a statistically significant performance improvement from the incorporation of unimodal text data. Furthermore, we show that the training set of labelled memes can be reduced by 40% without reducing the performance of the downstream model.