# Deciphering Hate: Identifying Hateful Memes and Their Targets arXiv: https://arxiv.org/abs/2403.10829 ยท HTML: https://mlsanity.com/paper/2403.10829/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Builds on 28 of its 52 references that are in the map. - First posted on arXiv 16 Mar 2024. ## Builds on (most cited first, 28 of 28) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021) - [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022) - [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019) - [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019) - [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021) - [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019) - [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020) - [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021) - [Supervised Multimodal Bitransformers for Classifying Images and Text](https://mlsanity.com/paper/1909.02950/) (2019) - [Exploring Hate Speech Detection in Multimodal Publications](https://mlsanity.com/paper/1910.03814/) (2019) - [MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets](https://mlsanity.com/paper/2109.05184/) (2021) - [Detecting Harmful Memes and Their Targets](https://mlsanity.com/paper/2110.00413/) (2021) - [Prompting for Multimodal Hateful Meme Classification](https://mlsanity.com/paper/2302.04156/) (2023) - [Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Features](https://mlsanity.com/paper/2210.05916/) (2022) - [Disentangling Hate in Online Memes](https://mlsanity.com/paper/2108.06207/) (2021) - [Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation](https://mlsanity.com/paper/1910.02334/) (2019) - [Multimodal Learning for Hateful Memes Detection](https://mlsanity.com/paper/2011.12870/) (2020) - [Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization](https://mlsanity.com/paper/2101.11075/) (2021) - [BanglaAbuseMeme: A Dataset for Bengali Abusive Meme Classification](https://mlsanity.com/paper/2310.11748/) (2023) - [Multimodal Hate Speech Detection from Bengali Memes and Texts](https://mlsanity.com/paper/2204.10196/) (2022) - [Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim?](https://mlsanity.com/paper/2301.11219/) (2023) - [AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Media](https://mlsanity.com/paper/2106.11229/) (2021) ## Built on by (most cited first, 7 of 7) - [Propaganda to Hate: A Multimodal Analysis of Arabic Memes with Multi-Agent LLMs](https://mlsanity.com/paper/2409.07246/) (2024) - [Retrieval Augmented Enhanced Dual Co-Attention Framework for Target Aware Multimodal Bengali Hateful Meme Detection](https://mlsanity.com/paper/2602.19212/) (2026) - [Detecting Hate and Inflammatory Content in Bengali Memes: A New Multimodal Dataset and Co-Attention Framework](https://mlsanity.com/paper/2602.22391/) (2026) - [HCIG: A Hierarchical Cross-Modal Incongruity Graph Network for Multimodal Sarcasm and Cyberbullying Detection](https://mlsanity.com/paper/2607.16076/) (2026) - [Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges](https://mlsanity.com/paper/2607.19011/) (2026) - [Token-Region Guided Cross-Attention Fusion for Multimodal Affect Interpretation](https://mlsanity.com/paper/2607.23493/) (2026) - [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026) ## Abstract (arXiv) Internet memes have become a powerful means for individuals to express emotions, thoughts, and perspectives on social media. While often considered as a source of humor and entertainment, memes can also disseminate hateful content targeting individuals or communities. Most existing research focuses on the negative aspects of memes in high-resource languages, overlooking the distinctive challenges associated with low-resource languages like Bengali (also known as Bangla). Furthermore, while previous work on Bengali memes has focused on detecting hateful memes, there has been no work on detecting their targeted entities. To bridge this gap and facilitate research in this arena, we introduce a novel multimodal dataset for Bengali, BHM (Bengali Hateful Memes). The dataset consists of 7,148 memes with Bengali as well as code-mixed captions, tailored for two tasks: (i) detecting hateful memes, and (ii) detecting the social entities they target (i.e., Individual, Organization, Community, and Society). To solve these tasks, we propose DORA (Dual cO attention fRAmework), a multimodal deep neural network that systematically extracts the significant modality features from the memes and jointly evaluates them with the modality-specific features to understand the context better. Our experiments show that DORA is generalizable on other low-resource hateful meme datasets and outperforms several state-of-the-art rivaling baselines.