# Benchmark dataset of memes with text transcriptions for automatic detection of multi-modal misogynistic content

arXiv: https://arxiv.org/abs/2106.08409 · HTML: https://mlsanity.com/paper/2106.08409/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 12 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 1 of 1)

- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)

## Built on by (most cited first, 9 of 9)

- [MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models](https://mlsanity.com/paper/2406.07594/) (2024)
- [Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models](https://mlsanity.com/paper/2401.16727/) (2024)
- [Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities](https://mlsanity.com/paper/2406.07353/) (2024)
- [M-QUEST -- Meme Question-Understanding Evaluation on Semantics and Toxicity](https://mlsanity.com/paper/2603.03315/) (2026)
- [Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models](https://mlsanity.com/paper/2502.18101/) (2025)
- [Unveiling Covert Toxicity in Multimodal Data via Toxicity Association Graphs: A Graph-Based Metric and Interpretable Detection Framework](https://mlsanity.com/paper/2602.03268/) (2026)
- [Distinguishing Right from Wrong in Debates: Attribution Analysis of Chinese Harmful Memes](https://mlsanity.com/paper/2605.24344/) (2026)
- [MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models](https://mlsanity.com/paper/2609.01772/) (2026)
- [Decodable but Misrouted: Sparse Features Uncover a Readout Gap in Vision-Language Models for Harmful Meme Detection](https://mlsanity.com/paper/2609.18860/) (2026)

## Abstract (arXiv)

In this paper we present a benchmark dataset generated as part of a project for automatic identification of misogyny within online content, which focuses in particular on memes. The benchmark here described is composed of 800 memes collected from the most popular social media platforms, such as Facebook, Twitter, Instagram and Reddit, and consulting websites dedicated to collection and creation of memes. To gather misogynistic memes, specific keywords that refer to misogynistic content have been considered as search criterion, considering different manifestations of hatred against women, such as body shaming, stereotyping, objectification and violence. In parallel, memes with no misogynist content have been manually downloaded from the same web sources. Among all the collected memes, three domain experts have selected a dataset of 800 memes equally balanced between misogynistic and non-misogynistic ones. This dataset has been validated through a crowdsourcing platform, involving 60 subjects for the labelling process, in order to collect three evaluations for each instance. Two further binary labels have been collected from both the experts and the crowdsourcing platform, for memes evaluated as misogynistic, concerning aggressiveness and irony. Finally for each meme, the text has been manually transcribed. The dataset provided is thus composed of the 800 memes, the labels given by the experts and those obtained by the crowdsourcing validation, and the transcribed texts. This data can be used to approach the problem of automatic detection of misogynistic content on the Web relying on both textual and visual cues, facing phenomenons that are growing every day such as cybersexism and technology-facilitated violence.