NYCU-TWO at Memotion 3: Good Foundation, Good Teacher, then you have Good Meme Analysis
Not on the map. Why?
Abstract (arXiv)
This paper presents a robust solution to the Memotion 3.0 Shared Task. The goal of this task is to classify the emotion and the corresponding intensity expressed by memes, which are usually in the form of images with short captions on social media. Understanding the multi-modal features of the given memes will be the key to solving the task. In this work, we use CLIP to extract aligned image-text features and propose a novel meme sentiment analysis framework, consisting of a Cooperative Teaching Model (CTM) for Task A and a Cascaded Emotion Classifier (CEC) for Tasks B&C. CTM is based on the idea of knowledge distillation, and can better predict the sentiment of a given meme in Task A; CEC can leverage the emotion intensity suggestion from the prediction of Task C to classify the emotion more precisely in Task B. Experiments show that we achieved the 2nd place ranking for both Task A and Task B and the 4th place ranking for Task C, with weighted F1-scores of 0.342, 0.784, and 0.535 respectively. The results show the robustness and effectiveness of our framework. Our code is released at github.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 26 references that are in the map. First posted on arXiv 13 Feb 2023.
Builds on
17 of 26 references.
References · 17
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
- 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
- 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
- 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
5 more
- 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
- 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
- 2020Extending Multilingual BERT to Low-Resource Languagescited by 88
- 2018Multimodal Sentiment Analysis To Explore the Structure of Emotionscited by 19
- 2022BLUE at Memotion 2.0 2022: You have my Image, my Text and my Transformercited by 5
Built on by
Citing papers per year: 2023: 1.
1 paper.