mlsanityData to 7 Oct 2026

Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models

arXiv 2502.18101 ↗First posted 25 Feb 2025cs.CV cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Traditional online content moderation systems struggle to classify modern multimodal means of communication, such as memes, a highly nuanced and information-dense medium. This task is especially hard in a culturally diverse society like Singapore, where low-resource languages are used and extensive knowledge on local context is needed to interpret online content. We curate a large collection of 112K memes labeled by GPT-4V for fine-tuning a VLM to classify offensive memes in Singapore context. We show the effectiveness of fine-tuned VLMs on our dataset, and propose a pipeline containing OCR, translation and a 7-billion parameter-class VLM. Our solutions reach 80.62% accuracy and 0.8192 AUROC on a held-out test set, and can greatly aid human in moderating online contents. The dataset, code, and model weights have been open-sourced at https://github.com/aliencaocao/vlm-for-memes-aisg.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 48 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

13 of 48 references.

References · 13

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  5. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  6. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 365
  7. 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 231
  8. 2022PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR Systemcited by 105
  9. 2021Disentangling Hate in Online Memescited by 46
  10. 2023MemeCap: A Dataset for Captioning and Interpreting Memescited by 46
  11. 2023Decoding the Underlying Meaning of Multimodal Hateful Memescited by 38
  12. 2021Benchmark dataset of memes with text transcriptions for automatic detection of multi-modal misogynistic contentcited by 9
1 more
  1. 2024LionGuard: Building a Contextualized Moderation Classifier to Tackle Localized Unsafe Contentcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Native Memes to Global Moderation: Cross-Cultural Evaluation of Vision-Language Models for Hateful Meme Detection
  2. 2024Exploring the Limits of Zero Shot Vision Language Models for Hate Meme Detection: The Vulnerabilities and their Interpretations
  3. 2023A Review of Vision-Language Models and their Performance on the Hateful Memes Challenge
  4. 2025On VLMs for Diverse Tasks in Multimodal Meme Classification
  5. 2023Detecting and Correcting Hate Speech in Multimodal Memes with Large Visual Language Model
  6. 2026MemeLens: Multilingual Multitask VLMs for Memes
  7. 2025Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Models
  8. 2024OSPC: Artificial VLM Features for Hateful Meme Detection