# Meme Similarity and Emotion Detection using Multimodal Analysis

arXiv: https://arxiv.org/abs/2503.17493 · HTML: https://mlsanity.com/paper/2503.17493/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 63 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021)
- [PointCLIP: Point Cloud Understanding by CLIP](https://mlsanity.com/paper/2112.02413/) (2021)
- [Long-CLIP: Unlocking the Long-Text Capability of CLIP](https://mlsanity.com/paper/2403.15378/) (2024)
- [Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Features](https://mlsanity.com/paper/2210.05916/) (2022)
- [CLIP-Nav: Using CLIP for Zero-Shot Vision-and-Language Navigation](https://mlsanity.com/paper/2211.16649/) (2022)
- [On the Origins of Memes by Means of Fringe Web Communities](https://mlsanity.com/paper/1805.12512/) (2018)
- [On the Evolution of (Hateful) Memes by Means of Multimodal Contrastive Learning](https://mlsanity.com/paper/2212.06573/) (2022)
- [Mapping Memes to Words for Multimodal Hateful Meme Classification](https://mlsanity.com/paper/2310.08368/) (2023)
- [Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models](https://mlsanity.com/paper/2312.05434/) (2023)
- [Predicting Successful Memes using Network and Community Structure](https://mlsanity.com/paper/1403.6199/) (2014)
- [Emotion-Aware Multimodal Fusion for Meme Emotion Detection](https://mlsanity.com/paper/2403.10279/) (2024)
- [Enhancing Image Retrieval : A Comprehensive Study on Photo Search using the CLIP Mode](https://mlsanity.com/paper/2401.13613/) (2024)
- [Multimodal Analysis of memes for sentiment extraction](https://mlsanity.com/paper/2112.11850/) (2021)
- [Towards Building an Open-Domain Dialogue System Incorporated with Internet Memes](https://mlsanity.com/paper/2203.03835/) (2022)
- [Social Meme-ing: Measuring Linguistic Variation in Memes](https://mlsanity.com/paper/2311.09130/) (2023)
- [Contextualizing Internet Memes Across Social Media Platforms](https://mlsanity.com/paper/2311.11157/) (2023)
- [PromptMTopic: Unsupervised Multimodal Topic Modeling of Memes using Large Language Models](https://mlsanity.com/paper/2312.06093/) (2023)
- [MemeTector: Enforcing deep focus for meme detection](https://mlsanity.com/paper/2205.13268/) (2022)
- [Clustering Memes in Social Media](https://mlsanity.com/paper/1310.2665/) (2013)
- [Semantic Search of Memes on Twitter](https://mlsanity.com/paper/2002.01462/) (2020)
- [SemiMemes: A Semi-supervised Learning Approach for Multimodal Memes Analysis](https://mlsanity.com/paper/2304.00020/) (2023)
- [Tracking Emotional Dynamics in Chat Conversations: A Hybrid Approach using DistilBERT and Emoji Sentiment Analysis](https://mlsanity.com/paper/2408.01838/) (2024)
- [Detection and Analysis of Stress-Related Posts in Reddit Acamedic Communities](https://mlsanity.com/paper/2312.01050/) (2023)

## Built on by (most cited first, 1 of 1)

- [Extending a Parliamentary Corpus with MPs' Tweets: Automatic Annotation and Evaluation Using MultiParTweet](https://mlsanity.com/paper/2512.11567/) (2025)

## Abstract (arXiv)

Internet memes are a central element of online culture, blending images and text. While substantial research has focused on either the visual or textual components of memes, little attention has been given to their interplay. This gap raises a key question: What methodology can effectively compare memes and the emotions they elicit? Our study employs a multimodal methodological approach, analyzing both the visual and textual elements of memes. Specifically, we perform a multimodal CLIP (Contrastive Language-Image Pre-training) model for grouping similar memes based on text and visual content embeddings, enabling robust similarity assessments across modalities. Using the Reddit Meme Dataset and Memotion Dataset, we extract low-level visual features and high-level semantic features to identify similar meme pairs. To validate these automated similarity assessments, we conducted a user study with 50 participants, asking them to provide yes/no responses regarding meme similarity and their emotional reactions. The comparison of experimental results with human judgments showed a 67.23% agreement, suggesting that the computational approach aligns well with human perception. Additionally, we implemented a text-based classifier using the DistilBERT model to categorize memes into one of six basic emotions. The results indicate that anger and joy are the dominant emotions in memes, with motivational memes eliciting stronger emotional responses. This research contributes to the study of multimodal memes, enhancing both language-based and visual approaches to analyzing and improving online visual communication and user experiences. Furthermore, it provides insights for better content moderation strategies in online platforms.