mlsanityData to 14 Oct 2026

Towards Low-Resource Harmful Meme Detection with LMM Agents

arXiv 2411.05383 ↗First posted 8 Nov 2024cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

The proliferation of Internet memes in the age of social media necessitates effective identification of harmful ones. Due to the dynamic nature of memes, existing data-driven models may struggle in low-resource scenarios where only a few labeled examples are available. In this paper, we propose an agency-driven framework for low-resource harmful meme detection, employing both outward and inward analysis with few-shot annotated samples. Inspired by the powerful capacity of Large Multimodal Models (LMMs) on multimodal reasoning, we first retrieve relative memes with annotations to leverage label information as auxiliary signals for the LMM agent. Then, we elicit knowledge-revising behavior within the LMM agent to derive well-generalized insights into meme harmfulness. By combining these strategies, our approach enables dialectical reasoning over intricate and implicit harm-indicative patterns. Extensive experiments conducted on three meme datasets demonstrate that our proposed approach achieves superior performance than state-of-the-art methods on the low-resource harmful meme detection task.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 54 of its 73 references that are in the map. First posted on arXiv 8 Nov 2024.

Builds on

54 of 73 references.

References · 54

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  8. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  9. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  10. 2023Segment Anythingcited by 8,234
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
38 more
  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  8. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  10. 2023Teaching Large Language Models to Self-Debugcited by 988
  11. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  12. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  13. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  14. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  15. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  16. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  17. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  18. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 490
  19. 2022CodeT: Code Generation with Generated Testscited by 384
  20. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 279
  21. 2023AdaPlanner: Adaptive Planning from Feedback with Language Modelscited by 196
  22. 2019Supervised Multimodal Bitransformers for Classifying Images and Textcited by 130
  23. 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 86
  24. 2021Detecting Harmful Memes and Their Targetscited by 78
  25. 2023Prompting for Multimodal Hateful Meme Classificationcited by 69
  26. 2022Detecting and Understanding Harmful Memes: A Surveycited by 58
  27. 2024GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abusecited by 52
  28. 2020Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memescited by 51
  29. 2020Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challengecited by 51
  30. 2020A Multimodal Framework for the Detection of Hateful Memescited by 49
  31. 2023Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detectioncited by 47
  32. 2021Disentangling Hate in Online Memescited by 46
  33. 2020Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solutioncited by 41
  34. 2022Detect Rumors in Microblog Posts for Low-Resource Domains via Adversarial Contrastive Learningcited by 41
  35. 2021MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understandingcited by 40
  36. 2020Detecting Hate Speech in Multi-modal Memescited by 38
  37. 2022Zero-Shot Rumor Detection with Propagation Structure via Prompt Learningcited by 32
  38. 2020Detecting Hateful Memes Using a Multimodal Deep Ensemblecited by 30

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 10.

24 papers.

Sort:

By area:

  1. 2025MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detectioncited by 15
  2. 2025Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detectioncited by 12
  3. 2025ExPO-HM: Learning to Explain-then-Detect for Hateful Meme Detectioncited by 8
  4. 2025Read as You See: Guiding Unimodal LLMs for Low-Resource Explainable Harmful Meme Detectioncited by 7
  5. 2025MemeReaCon: Probing Contextual Meme Understanding in Large Vision-Language Modelscited by 4
  6. 2025From Recognition to Reasoning: Advancing Multimodal Harmful Meme Detection via Chain-of-Thought Alignmentcited by 4
  7. 2026PEST: Parameter Efficient Steering of Blackbox VLMs via Agentic Few-shot Alignment for Hateful Meme Moderationcited by 4
  8. 2025What Media Frames Reveal About Stance: A Dataset and Study about Memes in Climate Change Discoursecited by 3
  9. 2026Yes FLoReNce, I Will Do Better Next Time! Agentic Feedback Reasoning for Humorous Meme Detectioncited by 3
  10. 2025MM-CRITIC: A Holistic Evaluation of Large Multimodal Models as Multimodal Critiquecited by 2
  11. 2026Robust Harmful Meme Detection under Missing Modalities via Shared Representation Learningcited by 2
  12. 2025Multi-Agent Synergy-Driven Iterative Visual Narrative Synthesiscited by 1
12 more
  1. 2025From Shallow Humor to Metaphor: Towards Label-Free Harmful Meme Detection via LMM Agent Self-Improvementcited by 1
  2. 2026All Changes May Have Invariant Principles: Improving Ever-Shifting Harmful Meme Detection via Design Concept Reproductioncited by 1
  3. 2025Selective Demonstration Retrieval for Improved Implicit Hate Speech Detectioncited by 0
  4. 2025Fall into a Pit, Gain in a Wit: Cognitive-Guided Harmful Meme Detection via Misjudgment Risk Pattern Retrievalcited by 0
  5. 2025A Multimodal Benchmark for Framing of Oil & Gas Advertising and Potential Greenwashing Detectioncited by 0
  6. 2025MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Modelscited by 0
  7. 2026PrismAgent: Illuminating Harm in Memes via a Zero-Shot Interpretable Multi-Agent Frameworkcited by 0
  8. 2026FigSIM: A Dataset for Fine-grained Suicide Severity and Figurative Language in Suicide Memescited by 0
  9. 2026Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memescited by 0
  10. 2026SyRHM: Symbolic-Language-Enhanced Reasoning with Associative Retrieval for Zero-shot Harmful Meme Detectioncited by 0
  11. 2026Learn Before You Judge: Progressive Knowledge-to-Decision Alignment for Explainable Hateful Meme Detectioncited by 0
  12. 2026MemeTrace: A Multimodal LLM-Based Agent for Safety-Aware Meme Provenance Analysiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models
  2. 2024Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Models
  3. 2025AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness
  4. 2025Improving Multimodal Hateful Meme Detection Exploiting LMM-Generated Knowledge
  5. 2026HarmTrace: Anchor-Calibrated Decoupled Optimization for Fine-Grained Target Identification in Harmful Memes
  6. 2026KID: Knowledge-Injected Dual-Head Learning for Knowledge-Grounded Harmful Meme Detection
  7. 2025Demystifying Hateful Content: Leveraging Large Multimodal Models for Hateful Meme Detection with Explainable Decisions
  8. 2024SAFE-MEME: Structured Reasoning Framework for Robust Hate Speech Detection in Memes