# Read as You See: Guiding Unimodal LLMs for Low-Resource Explainable Harmful Meme Detection

arXiv: https://arxiv.org/abs/2506.08477 · HTML: https://mlsanity.com/paper/2506.08477/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 83 references that are in the map.
- First posted on arXiv 10 Jun 2025.

## Builds on (most cited first, 30 of 31)

- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [A Survey on the Explainability of Supervised Machine Learning](https://mlsanity.com/paper/2011.07876/) (2020)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- [Detecting Harmful Memes and Their Targets](https://mlsanity.com/paper/2110.00413/) (2021)
- [Prompting for Multimodal Hateful Meme Classification](https://mlsanity.com/paper/2302.04156/) (2023)
- [Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Features](https://mlsanity.com/paper/2210.05916/) (2022)
- [On the Origins of Memes by Means of Fringe Web Communities](https://mlsanity.com/paper/1805.12512/) (2018)
- [Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection](https://mlsanity.com/paper/2308.08088/) (2023)
- [Effective Neural Topic Modeling with Embedding Clustering Regularization](https://mlsanity.com/paper/2306.04217/) (2023)
- [Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning](https://mlsanity.com/paper/2211.12878/) (2022)
- [AKEW: Assessing Knowledge Editing in the Wild](https://mlsanity.com/paper/2402.18909/) (2024)
- [MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification](https://mlsanity.com/paper/2409.14703/) (2024)
- [Towards Low-Resource Harmful Meme Detection with LMM Agents](https://mlsanity.com/paper/2411.05383/) (2024)
- [Towards the TopMost: A Topic Modeling System Toolkit](https://mlsanity.com/paper/2309.06908/) (2023)
- [Mapping Memes to Words for Multimodal Hateful Meme Classification](https://mlsanity.com/paper/2310.08368/) (2023)
- [MemeGuard: An LLM and VLM-based Framework for Advancing Content Moderation via Meme Intervention](https://mlsanity.com/paper/2406.05344/) (2024)
- [Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Models](https://mlsanity.com/paper/2401.13298/) (2024)
- [Modularized Networks for Few-shot Hateful Meme Detection](https://mlsanity.com/paper/2402.11845/) (2024)
- [Demystifying Hateful Content: Leveraging Large Multimodal Models for Hateful Meme Detection with Explainable Decisions](https://mlsanity.com/paper/2502.11073/) (2025)
- [Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities](https://mlsanity.com/paper/2406.07353/) (2024)
- [Towards Comprehensive Detection of Chinese Harmful Memes](https://mlsanity.com/paper/2410.02378/) (2024)
- [MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencing](https://mlsanity.com/paper/2405.11215/) (2024)
- [Are LLMs Good Zero-Shot Fallacy Classifiers?](https://mlsanity.com/paper/2410.15050/) (2024)
- [M3Hop-CoT: Misogynous Meme Identification with Multimodal Multi-hop Chain-of-Thought](https://mlsanity.com/paper/2410.09220/) (2024)
- [Contextualizing Internet Memes Across Social Media Platforms](https://mlsanity.com/paper/2311.11157/) (2023)
- [Towards Safer Social Media Platforms: Scalable and Performant Few-Shot Harmful Content Moderation Using Large Language Models](https://mlsanity.com/paper/2501.13976/) (2025)
- [MemeSense: An Adaptive In-Context Framework for Social Commonsense Driven Meme Moderation](https://mlsanity.com/paper/2502.11246/) (2025)

## Built on by (most cited first, 7 of 7)

- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [From Recognition to Reasoning: Advancing Multimodal Harmful Meme Detection via Chain-of-Thought Alignment](https://mlsanity.com/paper/2506.18919/) (2025)
- [PEST: Parameter Efficient Steering of Blackbox VLMs via Agentic Few-shot Alignment for Hateful Meme Moderation](https://mlsanity.com/paper/2601.04692/) (2026)
- [MuPHI: Learning Implicit Multimodal Harm Reasoning via Semantically Grounded Reward Optimization](https://mlsanity.com/paper/2605.29951/) (2026)
- [FBHM: Functional Benchmarking and Steering of VLMs for Hateful Meme Detection](https://mlsanity.com/paper/2605.31349/) (2026)
- [Learn Before You Judge: Progressive Knowledge-to-Decision Alignment for Explainable Hateful Meme Detection](https://mlsanity.com/paper/2609.19778/) (2026)
- [MemeLogos: Knowledge-Driven Retrieval-Augmented Reasoning for Harmful Meme Detection](https://mlsanity.com/paper/2610.15463/) (2026)

## Abstract (arXiv)

Detecting harmful memes is crucial for safeguarding the integrity and harmony of online environments, yet existing detection methods are often resource-intensive, inflexible, and lacking explainability, limiting their applicability in assisting real-world web content moderation. We propose U-CoT+, a resource-efficient framework that prioritizes accessibility, flexibility and transparency in harmful meme detection by fully harnessing the capabilities of lightweight unimodal large language models (LLMs). Instead of directly prompting or fine-tuning large multimodal models (LMMs) as black-box classifiers, we avoid immediate reasoning over complex visual inputs but decouple meme content recognition from meme harmfulness analysis through a high-fidelity meme-to-text pipeline, which collaborates lightweight LMMs and LLMs to convert multimodal memes into natural language descriptions that preserve critical visual information, thus enabling text-only LLMs to "see" memes by "reading". Grounded in textual inputs, we further guide unimodal LLMs' reasoning under zero-shot Chain-of-Thoughts (CoT) prompting with targeted, interpretable, context-aware, and easily obtained human-crafted guidelines, thus providing accountable step-by-step rationales, while enabling flexible and efficient adaptation to diverse sociocultural criteria of harmfulness. Extensive experiments on seven benchmark datasets show that U-CoT+ achieves performance comparable to resource-intensive baselines, highlighting its effectiveness and potential as a scalable, explainable, and low-resource solution to support harmful meme detection.