mlsanityData to 7 Oct 2026

MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detection

arXiv 2507.06908 ↗First posted 9 Jul 2025cs.CL cs.AIcited by 14

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

The rapid expansion of memes on social media has highlighted the urgent need for effective approaches to detect harmful content. However, traditional data-driven approaches struggle to detect new memes due to their evolving nature and the lack of up-to-date annotated data. To address this issue, we propose MIND, a multi-agent framework for zero-shot harmful meme detection that does not rely on annotated data. MIND implements three key strategies: 1) We retrieve similar memes from an unannotated reference set to provide contextual information. 2) We propose a bi-directional insight derivation mechanism to extract a comprehensive understanding of similar memes. 3) We then employ a multi-agent debate mechanism to ensure robust decision-making through reasoned arbitration. Extensive experiments on three meme datasets demonstrate that our proposed framework not only outperforms existing zero-shot approaches but also shows strong generalization across different model architectures and parameter scales, providing a scalable solution for harmful meme detection. The code is available at https://github.com/destroy-lonely/MIND.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 44 of its 60 references that are in the map. First posted on arXiv 9 Jul 2025.

Builds on

44 of 60 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  11. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  12. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
32 more
  1. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  2. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  7. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  8. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  9. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  10. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  11. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
  12. 2023AdaPlanner: Adaptive Planning from Feedback with Language Modelscited by 193
  13. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 157
  14. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  15. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolutioncited by 132
  16. 2019Supervised Multimodal Bitransformers for Classifying Images and Textcited by 130
  17. 2024MARG: Multi-Agent Review Generation for Scientific Paperscited by 98
  18. 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 85
  19. 2021Detecting Harmful Memes and Their Targetscited by 77
  20. 2023Prompting for Multimodal Hateful Meme Classificationcited by 68
  21. 2022Detecting and Understanding Harmful Memes: A Surveycited by 58
  22. 2020Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challengecited by 51
  23. 2020Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memescited by 50
  24. 2020A Multimodal Framework for the Detection of Hateful Memescited by 48
  25. 2021Disentangling Hate in Online Memescited by 46
  26. 2023Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detectioncited by 45
  27. 2020Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solutioncited by 41
  28. 2020Detecting Hateful Memes Using a Multimodal Deep Ensemblecited by 30
  29. 2020Multimodal Learning for Hateful Memes Detectioncited by 29
  30. 2022On Explaining Multimodal Hateful Meme Detection Modelscited by 29
  31. 2022On the Evolution of (Hateful) Memes by Means of Multimodal Contrastive Learningcited by 26
  32. 2024Towards Low-Resource Harmful Meme Detection with LMM Agentscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 12.

14 papers.

Sort:

By area:

  1. 2025From Evidence to Verdict: An Agent-Based Forensic Framework for AI-Generated Image Detectioncited by 5
  2. 2026MemeLens: Multilingual Multitask VLMs for Memescited by 5
  3. 2026Yes FLoReNce, I Will Do Better Next Time! Agentic Feedback Reasoning for Humorous Meme Detectioncited by 3
  4. 2026I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memescited by 2
  5. 2025ADOPT: Adaptive Dependency-Guided Joint Prompt Optimization for Multi-Step LLM Pipelinescited by 1
  6. 2026All Changes May Have Invariant Principles: Improving Ever-Shifting Harmful Meme Detection via Design Concept Reproductioncited by 1
  7. 2026Can Thinking Models Think to Detect Hateful Memes?cited by 1
  8. 2026Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memescited by 1
  9. 2026Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigationcited by 0
  10. 2026PrismAgent: Illuminating Harm in Memes via a Zero-Shot Interpretable Multi-Agent Frameworkcited by 0
  11. 2026I Know What You Meme, Even If it Emerged Today: Understanding Evolving Memes through Open-World Knowledge Acquisitioncited by 0
  12. 2026Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memescited by 0
2 more
  1. 2026SyRHM: Symbolic-Language-Enhanced Reasoning with Associative Retrieval for Zero-shot Harmful Meme Detectioncited by 0
  2. 2026Learn Before You Judge: Progressive Knowledge-to-Decision Alignment for Explainable Hateful Meme Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Shallow Humor to Metaphor: Towards Label-Free Harmful Meme Detection via LMM Agent Self-Improvement
  2. 2025From Recognition to Reasoning: Advancing Multimodal Harmful Meme Detection via Chain-of-Thought Alignment
  3. 2026PEST: Parameter Efficient Steering of Blackbox VLMs via Agentic Few-shot Alignment for Hateful Meme Moderation
  4. 2025AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness
  5. 2023Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models
  6. 2025MV-Debate: Multi-view Agent Debate with Dynamic Reflection Gating for Multimodal Harmful Content Detection in Social Media
  7. 2024Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Models
  8. 2025Fall into a Pit, Gain in a Wit: Cognitive-Guided Harmful Meme Detection via Misjudgment Risk Pattern Retrieval