From Recognition to Reasoning: Advancing Multimodal Harmful Meme Detection via Chain-of-Thought Alignment
On the map: LLM reasoning → Reasoning benchmarks and evaluation
also in 1 more
Abstract (arXiv)
As a multimodal communication medium that integrates images and text, memes often convey implicit harmful content through metaphors, satire, and humor, making harmful meme detection a complex and challenging task. Although recent studies have achieved considerable progress in detection accuracy and model interpretability, large-scale, high-quality datasets for harmful memes remain scarce. Moreover, existing methods still exhibit notable limitations in identifying implicit risks and understanding fine-grained semantics. To address these challenges, we construct MemeMind, a large-scale dataset for harmful meme detection. MemeMind comprises a broad collection of publicly available memes and adopts a rigorous and comprehensive taxonomy of harmful content developed in accordance with widely recognized international standards and contemporary Internet contexts. In addition, the dataset provides detailed structured Chain-of-Thought (CoT) reasoning annotations to support fine-grained analysis of harmfulness, implicit intentions, and underlying semantics in memes. Building upon MemeMind, we further propose MemeGuard, a reasoning-oriented multimodal framework for harmful meme detection. MemeGuard employs a three-stage training strategy to progressively enhance the model's visual understanding, multimodal reasoning, and harmful content discrimination capabilities, thereby improving both detection accuracy and the interpretability of model decisions. Extensive experimental results demonstrate that MemeGuard outperforms existing state-of-the-art methods on the MemeMind dataset, providing a solid foundation for future research on harmful meme detection and multimodal content safety.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 31 of its 59 references that are in the map. First posted on arXiv 15 Jun 2025.
Builds on
31 of 59 references.
References · 31
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2024GPT-4o System Cardcited by 4,588
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2016Understanding deep learning requires rethinking generalizationcited by 1,674
- 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
- 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
19 more
- 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
- 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 86
- 2021Detecting Harmful Memes and Their Targetscited by 78
- 2023Prompting for Multimodal Hateful Meme Classificationcited by 69
- 2022Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Featurescited by 56
- 2023Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detectioncited by 47
- 2024MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classificationcited by 30
- 2019Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderationcited by 29
- 2022On Explaining Multimodal Hateful Meme Detection Modelscited by 29
- 2024Towards Low-Resource Harmful Meme Detection with LMM Agentscited by 24
- 2023Improving Hateful Meme Detection through Retrieval-Guided Contrastive Learningcited by 21
- 2022DISARM: Detecting the Victims Targeted by Harmful Memescited by 20
- 2023Mapping Memes to Words for Multimodal Hateful Meme Classificationcited by 20
- 2024Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Modelscited by 18
- 2018Cross-Dataset Adaptation for Visual Question Answeringcited by 17
- 2024Towards Comprehensive Detection of Chinese Harmful Memescited by 13
- 2025Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detectioncited by 12
- 2025Read as You See: Guiding Unimodal LLMs for Low-Resource Explainable Harmful Meme Detectioncited by 7
- 2025Detecting and Understanding Hateful Contents in Memes Through Captioning and Visual Question-Answeringcited by 2
Built on by
Citing papers per year: 2025: 2; 2026: 2.
4 papers.
- 2025Labels or Input? Rethinking Augmentation in Multimodal Hate Detectioncited by 0
- 2025Is GPT-4o mini Blinded by its Own Safety Filters? Exposing the Multimodal-to-Unimodal Bottleneck in Hate Speech Detectioncited by 0
- 2026Distinguishing Right from Wrong in Debates: Attribution Analysis of Chinese Harmful Memescited by 0
- 2026HarmTrace: Anchor-Calibrated Decoupled Optimization for Fine-Grained Target Identification in Harmful Memescited by 0
- 2026HarmTrace: Anchor-Calibrated Decoupled Optimization for Fine-Grained Target Identification in Harmful Memes17 Aug 2026
- 2026Distinguishing Right from Wrong in Debates: Attribution Analysis of Chinese Harmful Memes23 May 2026
- 2025Is GPT-4o mini Blinded by its Own Safety Filters? Exposing the Multimodal-to-Unimodal Bottleneck in Hate Speech Detection17 Sep 2025
- 2025Labels or Input? Rethinking Augmentation in Multimodal Hate Detection15 Aug 2025
Similar, not cited
Close in wording, no citation either way
- 2023Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models
- 2025Fall into a Pit, Gain in a Wit: Cognitive-Guided Harmful Meme Detection via Misjudgment Risk Pattern Retrieval
- 2026Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memes
- 2026SyRHM: Symbolic-Language-Enhanced Reasoning with Associative Retrieval for Zero-shot Harmful Meme Detection
- 2024SAFE-MEME: Structured Reasoning Framework for Robust Hate Speech Detection in Memes
- 2025MemeBLIP2: A novel lightweight multimodal system to detect harmful memes
- 2026MemeLogos: Knowledge-Driven Retrieval-Augmented Reasoning for Harmful Meme Detection
- 2026KID: Knowledge-Injected Dual-Head Learning for Knowledge-Grounded Harmful Meme Detection