Unimodal Intermediate Training for Multimodal Meme Sentiment Classification
Not on the map. Why?
Abstract (arXiv)
Internet Memes remain a challenging form of user-generated content for automated sentiment classification. The availability of labelled memes is a barrier to developing sentiment classifiers of multimodal memes. To address the shortage of labelled memes, we propose to supplement the training of a multimodal meme classifier with unimodal (image-only and text-only) data. In this work, we present a novel variant of supervised intermediate training that uses relatively abundant sentiment-labelled unimodal data. Our results show a statistically significant performance improvement from the incorporation of unimodal text data. Furthermore, we show that the training set of labelled memes can be reduced by 40% without reducing the performance of the downstream model.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 45 references that are in the map. First posted on arXiv 1 Aug 2023.
Builds on
22 of 45 references.
References · 22
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
- 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
- 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
- 2018Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Taskscited by 214
- 2020Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?cited by 127
- 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 85
- 2020SemEval-2020 Task 8: Memotion Analysis -- The Visuo-Lingual Metaphor!cited by 77
- 2020DynaSent: A Dynamic Benchmark for Sentiment Analysiscited by 63
- 2021What to Pre-Train on? Efficient Intermediate Task Selectioncited by 62
- 2022Transferability in Deep Learning: A Surveycited by 57
- 2020Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memescited by 50
10 more
- 2020Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solutioncited by 41
- 2022Large-Scale Hate Speech Detection with Cross-Domain Transfercited by 39
- 2021Caption Enriched Samples for Improving Hateful Memes Detectioncited by 14
- 2021AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Mediacited by 13
- 2020IITK at SemEval-2020 Task 8: Unimodal and Bimodal Sentiment Analysis of Internet Memescited by 12
- 2023Memotion 3: Dataset on Sentiment and Emotion Analysis of Codemixed Hindi-English Memescited by 12
- 2021Exercise? I thought you said 'Extra Fries': Leveraging Sentence Demarcations and Multi-hop Attention for Meme Affect Analysiscited by 9
- 2022BLUE at Memotion 2.0 2022: You have my Image, my Text and my Transformercited by 5
- 2022Domain-aware Self-supervised Pre-training for Label-Efficient Meme Analysiscited by 2
- 2023Meme Sentiment Analysis Enhanced with Multimodal Spatial Encoding and Facial Embeddingcited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.