mlsanityData to 14 Oct 2026

Exploring Hate Speech Detection in Multimodal Publications

arXiv 1910.03814 ↗First posted 9 Oct 2019cs.CV cs.CLcited by 103

Not on the map. Why?

Abstract (arXiv)

In this work we target the problem of hate speech detection in multimodal publications formed by a text and an image. We gather and annotate a large scale dataset from Twitter, MMHS150K, and propose different models that jointly analyze textual and visual information for hate speech detection, comparing them with unimodal detection. We provide quantitative and qualitative results and analyze the challenges of the proposed task. We find that, even though images are useful for the hate speech detection task, current multimodal models cannot outperform models analyzing only text. We discuss why and open the field and the dataset for further research.

Key facts. Cited by 103 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 12; 2023: 17; 2024: 24; 2025: 16; 2026: 18.

103 papers.

Sort:

By area:

  1. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
  2. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 155
  3. 2023Prompting for Multimodal Hateful Meme Classificationcited by 69
  4. 2022Detecting and Understanding Harmful Memes: A Surveycited by 58
  5. 2021A Survey on Multimodal Disinformation Detectioncited by 47
  6. 2021Disentangling Hate in Online Memescited by 46
  7. 2023HateMM: A Multi-Modal Dataset for Hate Video Classificationcited by 42
  8. 2022On Explaining Multimodal Hateful Meme Detection Modelscited by 29
  9. 2021The MuSe 2021 Multimodal Sentiment Analysis Challenge: Sentiment, Emotion, Physiological-Emotion, and Stresscited by 27
  10. 2021A systematic review of Hate Speech automatic detection using Natural Language Processingcited by 27
  11. 2024Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Modelscited by 26
  12. 2021The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvementscited by 22
38 more
  1. 2021"Subverting the Jewtocracy": Online Antisemitism Detection Using Multimodal Deep Learningcited by 21
  2. 2022DISARM: Detecting the Victims Targeted by Harmful Memescited by 20
  3. 2023BanglaAbuseMeme: A Dataset for Bengali Abusive Meme Classificationcited by 18
  4. 2023LAraBench: Benchmarking Arabic AI with Large Language Modelscited by 16
  5. 2022Multimodal Hate Speech Detection from Bengali Memes and Textscited by 15
  6. 2023Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim?cited by 14
  7. 2022Perturbations in the Wild: Leveraging Human-Written Text Perturbations for Realistic Adversarial Attack and Defensecited by 13
  8. 2024Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanationscited by 13
  9. 2024Towards Comprehensive Detection of Chinese Harmful Memescited by 13
  10. 2021Exploring Visual Engagement Signals for Representation Learningcited by 12
  11. 2023Memotion 3: Dataset on Sentiment and Emotion Analysis of Codemixed Hindi-English Memescited by 12
  12. 2024SoMeLVLM: A Large Vision Language Model for Social Media Processingcited by 11
  13. 2024Exploring the Limits of Zero Shot Vision Language Models for Hate Meme Detection: The Vulnerabilities and their Interpretationscited by 10
  14. 2024ToxVidLM: A Multimodal Framework for Toxicity Detection in Code-Mixed Videoscited by 10
  15. 2024BanTH: A Multi-label Hate Speech Detection Dataset for Transliterated Banglacited by 10
  16. 2021Multimodal Fusion Refiner Networkscited by 9
  17. 2023Rethinking Multimodal Content Moderation from an Asymmetric Angle with Mixed-modalitycited by 9
  18. 2025Towards a Robust Framework for Multimodal Hate Detection: A Study on Video vs. Image-based Contentcited by 9
  19. 2021Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hatecited by 8
  20. 2021Revisiting Contextual Toxicity Detection in Conversationscited by 8
  21. 2025Enhanced Multimodal Hate Video Detection via Channel-wise and Modality-wise Fusioncited by 8
  22. 2024Deciphering Hate: Identifying Hateful Memes and Their Targetscited by 7
  23. 2024Multi3Hate: Multimodal, Multilingual, and Multicultural Hate Speech Detection with Vision-Language Modelscited by 7
  24. 2023CRYPTEXT: Database and Interactive Toolkit of Human-Written Text Perturbations in the Wildcited by 6
  25. 2026MemeLens: Multilingual Multitask VLMs for Memescited by 6
  26. 2021Exploring multi-task multi-lingual learning of transformer models for hate speech and offensive speech identification in social mediacited by 5
  27. 2021Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemblecited by 5
  28. 2022BLUE at Memotion 2.0 2022: You have my Image, my Text and my Transformercited by 5
  29. 2024Neural Multimodal Topic Modeling: A Comprehensive Evaluationcited by 5
  30. 2024ArMeme: Propagandistic Content in Arabic Memescited by 5
  31. 2025The Impact of Persona-based Political Perspectives on Hateful Content Detectioncited by 5
  32. 2025MM-HSD: Multi-Modal Hate Speech Detection in Videoscited by 5
  33. 2021TweetBLM: A Hate Speech Dataset and Analysis of Black Lives Matter-related Microblogs on Twittercited by 4
  34. 2022TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memescited by 4
  35. 2024Align before Attend: Aligning Visual and Textual Features for Multimodal Hateful Content Detectioncited by 4
  36. 2024SAFE-MEME: Structured Reasoning Framework for Robust Hate Speech Detection in Memescited by 4
  37. 2025A Multimodal-Multitask Framework with Cross-modal Relation and Hierarchical Interactive Attention for Semantic Comprehensioncited by 4
  38. 2024GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection?cited by 3

50 most cited of 103 All 103 (JSON)