mlsanityData to 7 Oct 2026

PEST: Parameter Efficient Steering of Blackbox VLMs via Agentic Few-shot Alignment for Hateful Meme Moderation

arXiv 2601.04692 ↗First posted 8 Jan 2026cs.CL cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

In this work, we examine hateful memes from three complementary angles - how to detect them, how to explain their content and how to intervene them before being posted - by applying a range of strategies built on top of generative AI models. To the best of our knowledge, explanation and intervention have typically been studied separately from detection, which does not reflect real-world conditions. Further, since curating large annotated datasets for meme moderation is prohibitively expensive, we propose a novel framework - PEST - that leverages task-specific generative VLMs and the few-shot adaptability of large VLMs to cater to different types of memes. We believe this is the first work focused on generalizable hateful meme moderation under limited data conditions, and has strong potential for deployment in real-world production scenarios. Warning: Contains potentially toxic contents.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 8 Jan 2026.

Builds on

23 of 34 references.

References · 23

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  4. 2023Prompting for Multimodal Hateful Meme Classificationcited by 68
  5. 2018Thou shalt not hate: Countering Online Hate Speechcited by 64
  6. 2024GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abusecited by 50
  7. 2023MemeCap: A Dataset for Captioning and Interpreting Memescited by 46
  8. 2023Decoding the Underlying Meaning of Multimodal Hateful Memescited by 38
  9. 2022On Explaining Multimodal Hateful Meme Detection Modelscited by 29
  10. 2024Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Modelscited by 26
  11. 2022Few-Shot Continual Active Learning by a Robotcited by 24
  12. 2024Towards Low-Resource Harmful Meme Detection with LMM Agentscited by 23
11 more
  1. 2020Generalized Word Shift Graphs: A Method for Visualizing and Explaining Pairwise Comparisons Between Textscited by 21
  2. 2024MemeGuard: An LLM and VLM-based Framework for Advancing Content Moderation via Meme Interventioncited by 20
  3. 2023BanglaAbuseMeme: A Dataset for Bengali Abusive Meme Classificationcited by 18
  4. 2024Modularized Networks for Few-shot Hateful Meme Detectioncited by 15
  5. 2025Demystifying Hateful Content: Leveraging Large Multimodal Models for Hateful Meme Detection with Explainable Decisionscited by 13
  6. 2024MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencingcited by 12
  7. 2025MemeIntel: Explainable Detection of Propagandistic and Hateful Memescited by 8
  8. 2024Bridging Modalities: Enhancing Cross-Modality Hate Speech Detection with Few-Shot In-Context Learningcited by 7
  9. 2025Read as You See: Guiding Unimodal LLMs for Low-Resource Explainable Harmful Meme Detectioncited by 6
  10. 2025Toxicity Begets Toxicity: Unraveling Conversational Chains in Political Podcastscited by 4
  11. 2025MemeSense: An Adaptive In-Context Framework for Social Commonsense Driven Meme Moderationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026FBHM: Functional Benchmarking and Steering of VLMs for Hateful Meme Detectioncited by 0
  2. 2026I Know What You Meme, Even If it Emerged Today: Understanding Evolving Memes through Open-World Knowledge Acquisitioncited by 0
  3. 2026Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memescited by 0
  4. 2026Learn Before You Judge: Progressive Knowledge-to-Decision Alignment for Explainable Hateful Meme Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Is AI Ready for Multimodal Hate Speech Detection? A Comprehensive Dataset and Benchmark Evaluation ↗
  2. 2023A Review of Vision-Language Models and their Performance on the Hateful Memes Challenge
  3. 2025Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detection
  4. 2026Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memes
  5. 2025Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Models
  6. 2026Can Thinking Models Think to Detect Hateful Memes?
  7. 2025ExPO-HM: Learning to Explain-then-Detect for Hateful Meme Detection
  8. 2025Labels or Input? Rethinking Augmentation in Multimodal Hate Detection