mlsanityData to 7 Oct 2026

Prompting for Multimodal Hateful Meme Classification

arXiv 2302.04156 ↗First posted 8 Feb 2023cs.CL cs.IR cs.MMcited by 68

Not on the map. Why?

Abstract (arXiv)

Hateful meme classification is a challenging multimodal task that requires complex reasoning and contextual background knowledge. Ideally, we could leverage an explicit external knowledge base to supplement contextual and cultural information in hateful memes. However, there is no known explicit external knowledge base that could provide such hate speech contextual information. To address this gap, we propose PromptHate, a simple yet effective prompt-based model that prompts pre-trained language models (PLMs) for hateful meme classification. Specifically, we construct simple prompts and provide a few in-context examples to exploit the implicit knowledge in the pre-trained RoBERTa language model for hateful meme classification. We conduct extensive experiments on two publicly available hateful and offensive meme datasets. Our experimental results show that PromptHate is able to achieve a high AUC of 90.96, outperforming state-of-the-art baselines on the hateful meme classification task. We also perform fine-grained analyses and case studies on various prompt settings and demonstrate the effectiveness of the prompts on hateful meme classification.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 41 of its 47 references that are in the map. First posted on arXiv 8 Feb 2023.

Builds on

41 of 47 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  8. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  9. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  10. 2019Language Models as Knowledge Bases?cited by 1,922
  11. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  12. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
29 more
  1. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  2. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  3. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  4. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  5. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  6. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  7. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  8. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  9. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 298
  10. 2018A Simple Method for Commonsense Reasoningcited by 223
  11. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  12. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Modelscited by 182
  13. 2019oLMpics -- On what Language Model Pre-training Capturescited by 161
  14. 2019Supervised Multimodal Bitransformers for Classifying Images and Textcited by 130
  15. 2021KAT: A Knowledge Augmented Transformer for Vision-and-Languagecited by 125
  16. 2019Exploring Hate Speech Detection in Multimodal Publicationscited by 103
  17. 2019FairFace: Face Attribute Dataset for Balanced Race, Gender, and Agecited by 87
  18. 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 85
  19. 2021Detecting Harmful Memes and Their Targetscited by 77
  20. 2020Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challengecited by 51
  21. 2020Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memescited by 50
  22. 2020A Multimodal Framework for the Detection of Hateful Memescited by 48
  23. 2021Disentangling Hate in Online Memescited by 46
  24. 2020Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solutioncited by 41
  25. 2017The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Taskcited by 39
  26. 2020Detecting Hateful Memes Using a Multimodal Deep Ensemblecited by 30
  27. 2020Multimodal Learning for Hateful Memes Detectioncited by 29
  28. 2022On Explaining Multimodal Hateful Meme Detection Modelscited by 29
  29. 2020Hateful Memes Detection via Complementary Visual and Linguistic Networkscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 14; 2024: 18; 2025: 19; 2026: 17.

68 papers.

Sort:

By area:

  1. 2023Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detectioncited by 45
  2. 2023ICL-D3IE: In-Context Learning with Diverse Demonstrations Updating for Document Information Extractioncited by 38
  3. 2023Decoding the Underlying Meaning of Multimodal Hateful Memescited by 38
  4. 2024MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classificationcited by 29
  5. 2024Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Modelscited by 26
  6. 2024MultiHateClip: A Multilingual Benchmark Dataset for Hateful Video Detection on YouTube and Bilibilicited by 25
  7. 2024Towards Low-Resource Harmful Meme Detection with LMM Agentscited by 23
  8. 2023Improving Hateful Meme Detection through Retrieval-Guided Contrastive Learningcited by 20
  9. 2023Mapping Memes to Words for Multimodal Hateful Meme Classificationcited by 19
  10. 2024Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Modelscited by 17
  11. 2023Read, Diagnose and Chat: Towards Explainable and Interactive LLMs-Augmented Depression Detection in Social Mediacited by 16
  12. 2024Modularized Networks for Few-shot Hateful Meme Detectioncited by 15
38 more
  1. 2025MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detectioncited by 14
  2. 2024Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicitiescited by 13
  3. 2023TotalDefMeme: A Multi-Attribute Meme dataset on Total Defence in Singaporecited by 12
  4. 2023Debiasing Multimodal Sarcasm Detection with Contrastive Learningcited by 12
  5. 2024MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencingcited by 12
  6. 2024Towards Comprehensive Detection of Chinese Harmful Memescited by 12
  7. 2025ImpliHateVid: A Benchmark Dataset and Two-stage Contrastive Learning Framework for Implicit Hate Speech Detection in Videoscited by 12
  8. 2025Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detectioncited by 11
  9. 2024Exploring the Limits of Zero Shot Vision Language Models for Hate Meme Detection: The Vulnerabilities and their Interpretationscited by 10
  10. 2024MemeCraft: Contextual and Stance-Driven Multimodal Meme Generationcited by 9
  11. 2025Cross-Modal Transfer from Memes to Videos: Addressing Data Scarcity in Hateful Video Detectioncited by 9
  12. 2025MemeIntel: Explainable Detection of Propagandistic and Hateful Memescited by 8
  13. 2024Deciphering Hate: Identifying Hateful Memes and Their Targetscited by 7
  14. 2024Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memescited by 7
  15. 2023Multi-Modal Discussion Transformer: Integrating Text, Images and Graph Transformers to Detect Hate Speech on Social Mediacited by 6
  16. 2025Read as You See: Guiding Unimodal LLMs for Low-Resource Explainable Harmful Meme Detectioncited by 6
  17. 2023Contextualizing Internet Memes Across Social Media Platformscited by 5
  18. 2023PromptMTopic: Unsupervised Multimodal Topic Modeling of Memes using Large Language Modelscited by 5
  19. 2024ArMeme: Propagandistic Content in Arabic Memescited by 5
  20. 2025Humor in Pixels: Benchmarking Large Multimodal Models Understanding of Online Comicscited by 5
  21. 2026MemeLens: Multilingual Multitask VLMs for Memescited by 5
  22. 2024Align before Attend: Aligning Visual and Textual Features for Multimodal Hateful Content Detectioncited by 4
  23. 2024SAFE-MEME: Structured Reasoning Framework for Robust Hate Speech Detection in Memescited by 4
  24. 2026PEST: Parameter Efficient Steering of Blackbox VLMs via Agentic Few-shot Alignment for Hateful Meme Moderationcited by 4
  25. 2025What Media Frames Reveal About Stance: A Dataset and Study about Memes in Climate Change Discoursecited by 3
  26. 2025From Recognition to Reasoning: Advancing Multimodal Harmful Meme Detection via Chain-of-Thought Alignmentcited by 3
  27. 2026Yes FLoReNce, I Will Do Better Next Time! Agentic Feedback Reasoning for Humorous Meme Detectioncited by 3
  28. 2023Improving Multimodal Classification of Social Media Posts by Leveraging Image-Text Auxiliary Taskscited by 2
  29. 2024HateSieve: A Contrastive Learning Framework for Detecting and Segmenting Hateful Content in Multimodal Memescited by 2
  30. 2025Improving Multimodal Hateful Meme Detection Exploiting LMM-Generated Knowledgecited by 2
  31. 2025Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Modelscited by 2
  32. 2025On VLMs for Diverse Tasks in Multimodal Meme Classificationcited by 2
  33. 2025Cracking the Code: Enhancing Implicit Hate Speech Detection through Coding Classificationcited by 2
  34. 2025Representation Decomposition for Learning Similarity and Contrastness Across Modalities for Affective Computingcited by 2
  35. 2026From Native Memes to Global Moderation: Cross-Cultural Evaluation of Vision-Language Models for Hateful Meme Detectioncited by 2
  36. 2026MemeTAG: Keyword-Driven Meme Classification through Tag Embedding Reconstructioncited by 2
  37. 2023Explainable Multimodal Sentiment Analysis on Bengali Memescited by 1
  38. 2024Prompt-enhanced Network for Hateful Meme Classificationcited by 1

50 most cited of 68 All 68 (JSON)