# Multimodal Hate Speech Detection from Bengali Memes and Texts

arXiv: https://arxiv.org/abs/2204.10196 · HTML: https://mlsanity.com/paper/2204.10196/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 19 Apr 2022.

## Builds on (most cited first, 16 of 16)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [Exploring Hate Speech Detection in Multimodal Publications](https://mlsanity.com/paper/1910.03814/) (2019)
- [Multilingual Offensive Language Identification with Cross-lingual Embeddings](https://mlsanity.com/paper/2010.05324/) (2020)
- [Hate Lingo: A Target-based Linguistic Analysis of Hate Speech in Social Media](https://mlsanity.com/paper/1804.04257/) (2018)
- [Characterizing and Detecting Hateful Users on Twitter](https://mlsanity.com/paper/1803.08977/) (2018)
- [Hate Speech detection in the Bengali language: A dataset and its baseline evaluation](https://mlsanity.com/paper/2012.09686/) (2020)
- [Classification Benchmarks for Under-resourced Bengali Language based on Multichannel Convolutional-LSTM Network](https://mlsanity.com/paper/2004.07807/) (2020)
- [Deep Orthogonal Fusion: Multimodal Prognostic Biomarker Discovery Integrating Radiology, Pathology, Genomic, and Clinical Data](https://mlsanity.com/paper/2107.00648/) (2021)
- [DeepHateExplainer: Explainable Hate Speech Detection in Under-resourced Bengali Language](https://mlsanity.com/paper/2012.14353/) (2020)
- [Is a picture worth a thousand words? A Deep Multi-Modal Fusion Architecture for Product Classification in e-commerce](https://mlsanity.com/paper/1611.09534/) (2016)
- [Semantic Referee: A Neural-Symbolic Framework for Enhancing Geospatial Semantic Segmentation](https://mlsanity.com/paper/1904.13196/) (2019)
- [Multimodal Social Media Analysis for Gang Violence Prevention](https://mlsanity.com/paper/1807.08465/) (2018)

## Built on by (most cited first, 15 of 15)

- [BanglaAbuseMeme: A Dataset for Bengali Abusive Meme Classification](https://mlsanity.com/paper/2310.11748/) (2023)
- [Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes](https://mlsanity.com/paper/2411.12174/) (2024)
- [Deciphering Hate: Identifying Hateful Memes and Their Targets](https://mlsanity.com/paper/2403.10829/) (2024)
- [Multi3Hate: Multimodal, Multilingual, and Multicultural Hate Speech Detection with Vision-Language Models](https://mlsanity.com/paper/2411.03888/) (2024)
- [Large Multimodal Models for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2502.05568/) (2025)
- [BIDWESH: A Bangla Regional Based Hate Speech Detection Dataset](https://mlsanity.com/paper/2507.16183/) (2025)
- [Parameter-Efficient Fine-Tuning for Low-Resource Languages: A Comparative Study of LLMs for Bengali Hate Speech Detection](https://mlsanity.com/paper/2510.16985/) (2025)
- [A Survey on Automatic Online Hate Speech Detection in Low-Resource Languages](https://mlsanity.com/paper/2411.19017/) (2024)
- [Explainable Multimodal Sentiment Analysis on Bengali Memes](https://mlsanity.com/paper/2401.09446/) (2023)
- [AI-UPV at EXIST 2023 -- Sexism Characterization Using Large Language Models Under The Learning with Disagreements Regime](https://mlsanity.com/paper/2307.03385/) (2023)
- [Detecting Racist Text in Bengali: An Ensemble Deep Learning Framework](https://mlsanity.com/paper/2401.16748/) (2024)
- [BanglaMM-Disaster: A Multimodal Transformer-Based Deep Learning Framework for Multiclass Disaster Classification in Bangla](https://mlsanity.com/paper/2511.21364/) (2025)
- [BengaliSent140: A Large-Scale Bengali Binary Sentiment Dataset for Hate and Non-Hate Speech Classification](https://mlsanity.com/paper/2601.20129/) (2026)
- [Retrieval Augmented Enhanced Dual Co-Attention Framework for Target Aware Multimodal Bengali Hateful Meme Detection](https://mlsanity.com/paper/2602.19212/) (2026)
- [Detecting Hate and Inflammatory Content in Bengali Memes: A New Multimodal Dataset and Co-Attention Framework](https://mlsanity.com/paper/2602.22391/) (2026)

## Abstract (arXiv)

Numerous machine learning (ML) and deep learning (DL)-based approaches have been proposed to utilize textual data from social media for anti-social behavior analysis like cyberbullying, fake news detection, and identification of hate speech mainly for highly-resourced languages such as English. However, despite having a lot of diversity and millions of native speakers, some languages like Bengali are under-resourced, which is due to a lack of computational resources for natural language processing (NLP). Similar to other languages, Bengali social media contents also include images along with texts (e.g., multimodal memes are posted by embedding short texts into images on Facebook). Therefore, only the textual data is not enough to judge them since images might give extra context to make a proper judgement. This paper is about hate speech detection from multimodal Bengali memes and texts. We prepared the only multimodal hate speech dataset for-a-kind of problem for Bengali, which we use to train state-of-the-art neural architectures (e.g., Bi-LSTM/Conv-LSTM with word embeddings, ConvNets + pre-trained language models, e.g., monolingual Bangla BERT, multilingual BERT-cased/uncased, and XLM-RoBERTa) to jointly analyze textual and visual information for hate speech detection. Conv-LSTM and XLM-RoBERTa models performed best for texts, yielding F1 scores of 0.78 and 0.82, respectively. As of memes, ResNet-152 and DenseNet-161 models yield F1 scores of 0.78 and 0.79, respectively. As for multimodal fusion, XLM-RoBERTa + DenseNet-161 performed the best, yielding an F1 score of 0.83. Our study suggests that text modality is most useful for hate speech detection, while memes are moderately useful.