# MultiHateClip: A Multilingual Benchmark Dataset for Hateful Video Detection on YouTube and Bilibili

arXiv: https://arxiv.org/abs/2408.03468 · HTML: https://mlsanity.com/paper/2408.03468/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 28 Jul 2024.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling](https://mlsanity.com/paper/2102.06183/) (2021)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation](https://mlsanity.com/paper/2002.06353/) (2020)
- [Hate Speech Dataset from a White Supremacy Forum](https://mlsanity.com/paper/1809.04444/) (2018)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [Directions in Abusive Language Training Data: Garbage In, Garbage Out](https://mlsanity.com/paper/2004.01670/) (2020)
- [Understanding and Evaluating Racial Biases in Image Captioning](https://mlsanity.com/paper/2106.08503/) (2021)
- [COLD: A Benchmark for Chinese Offensive Language Detection](https://mlsanity.com/paper/2201.06025/) (2022)
- [Detecting Online Hate Speech Using Context Aware Models](https://mlsanity.com/paper/1710.07395/) (2017)
- [Prompting for Multimodal Hateful Meme Classification](https://mlsanity.com/paper/2302.04156/) (2023)
- [Detecting and Understanding Harmful Memes: A Survey](https://mlsanity.com/paper/2205.04274/) (2022)
- [Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection](https://mlsanity.com/paper/2308.08088/) (2023)
- [Facilitating Fine-grained Detection of Chinese Toxic Language: Hierarchical Taxonomy, Resources, and Benchmarks](https://mlsanity.com/paper/2305.04446/) (2023)
- [SWSR: A Chinese Dataset and Lexicon for Online Sexism Detection](https://mlsanity.com/paper/2108.03070/) (2021)
- [HateMM: A Multi-Modal Dataset for Hate Video Classification](https://mlsanity.com/paper/2305.03915/) (2023)
- [Decoding the Underlying Meaning of Multimodal Hateful Memes](https://mlsanity.com/paper/2305.17678/) (2023)
- [Survey: Transformer based Video-Language Pre-training](https://mlsanity.com/paper/2109.09920/) (2021)
- [Mono vs Multilingual BERT for Hate Speech Detection and Text Classification: A Case Study in Marathi](https://mlsanity.com/paper/2204.08669/) (2022)
- [Video Vision Transformers for Violence Detection](https://mlsanity.com/paper/2209.03561/) (2022)
- [What is BitChute? Characterizing the "Free Speech" Alternative to YouTube](https://mlsanity.com/paper/2004.01984/) (2020)

## Built on by (most cited first, 25 of 25)

- [Survey of Cultural Awareness in Language Models: Text and Beyond](https://mlsanity.com/paper/2411.00860/) (2024)
- [ImpliHateVid: A Benchmark Dataset and Two-stage Contrastive Learning Framework for Implicit Hate Speech Detection in Videos](https://mlsanity.com/paper/2508.06570/) (2025)
- [Cross-Modal Transfer from Memes to Videos: Addressing Data Scarcity in Hateful Video Detection](https://mlsanity.com/paper/2501.15438/) (2025)
- [Towards a Robust Framework for Multimodal Hate Detection: A Study on Video vs. Image-based Content](https://mlsanity.com/paper/2502.07138/) (2025)
- [MM-HSD: Multi-Modal Hate Speech Detection in Videos](https://mlsanity.com/paper/2508.20546/) (2025)
- [Towards Trustworthy Multimodal Moderation via Policy-Aligned Reasoning and Hierarchical Labeling](https://mlsanity.com/paper/2508.03296/) (2025)
- [RAVEN++: Pinpointing Fine-Grained Violations in Advertisement Videos with Active Reinforcement Reasoning](https://mlsanity.com/paper/2511.19168/) (2025)
- [HateClipSeg: A Segment-Level Annotated Dataset for Fine-Grained Hate Video Detection](https://mlsanity.com/paper/2508.01712/) (2025)
- [Revealing Temporal Label Noise in Multimodal Hateful Video Classification](https://mlsanity.com/paper/2508.04900/) (2025)
- [OutSafe-Bench: A Benchmark for Multimodal Offensive Content Detection in Large Language Models](https://mlsanity.com/paper/2511.10287/) (2025)
- [From Manipulation to Mistrust: Explaining Diverse Micro-Video Misinformation for Robust Debunking in the Wild](https://mlsanity.com/paper/2603.25423/) (2026)
- [Multimodal Hate Detection Using Dual-Stream Graph Neural Networks](https://mlsanity.com/paper/2509.13515/) (2025)
- [StreamSense: Streaming Social Task Detection with Selective Vision-Language Model Routing](https://mlsanity.com/paper/2601.22738/) (2026)
- [RB-FT: Rationale-Bootstrapped Fine-Tuning for Video Classification](https://mlsanity.com/paper/2511.15923/) (2025)
- [Shedding the Facades, Connecting the Domains: Detecting Shifting Multimodal Hate Video with Test-Time Adaptation](https://mlsanity.com/paper/2602.00132/) (2026)
- [Reasoning-Aware Multimodal Fusion for Hateful Video Detection](https://mlsanity.com/paper/2512.02743/) (2025)
- [TANDEM: Temporal-Aware Neural Detection for Multimodal Hate Speech](https://mlsanity.com/paper/2601.11178/) (2026)
- [Training-Free and Interpretable Hateful Video Detection via Multi-stage Adversarial Reasoning](https://mlsanity.com/paper/2601.15115/) (2026)
- [Towards Training-free Multimodal Hate Localisation with Large Language Models](https://mlsanity.com/paper/2602.09637/) (2026)
- [Beyond Binary Classification: Detecting Fine-Grained Sexism in Social Media Videos](https://mlsanity.com/paper/2602.15757/) (2026)
- [When Jokes Cross the Line: Analyzing Regular Humor and Dark Humor in YouTube Shorts](https://mlsanity.com/paper/2606.00046/) (2026)
- [Decoding Multimodal Cues: Unveiling the Implicit Meaning Behind Hateful Videos](https://mlsanity.com/paper/2606.11953/) (2026)
- [HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models](https://mlsanity.com/paper/2606.27187/) (2026)
- [CLARA: Clip-Level Multimodal Alignment with VLM-Derived Rationales for Hateful Video Detection](https://mlsanity.com/paper/2608.15905/) (2026)
- [MexHat: A Dataset for Hate Speech Detection in Mexican Spanish Videos](https://mlsanity.com/paper/2609.31553/) (2026)

## Abstract (arXiv)

Hate speech is a pressing issue in modern society, with significant effects both online and offline. Recent research in hate speech detection has primarily centered on text-based media, largely overlooking multimodal content such as videos. Existing studies on hateful video datasets have predominantly focused on English content within a Western context and have been limited to binary labels (hateful or non-hateful), lacking detailed contextual information. This study presents MultiHateClip1 , an novel multilingual dataset created through hate lexicons and human annotation. It aims to enhance the detection of hateful videos on platforms such as YouTube and Bilibili, including content in both English and Chinese languages. Comprising 2,000 videos annotated for hatefulness, offensiveness, and normalcy, this dataset provides a cross-cultural perspective on gender-based hate speech. Through a detailed examination of human annotation results, we discuss the differences between Chinese and English hateful videos and underscore the importance of different modalities in hateful and offensive video analysis. Evaluations of state-of-the-art video classification models, such as VLM, GPT-4V and Qwen-VL, on MultiHateClip highlight the existing challenges in accurately distinguishing between hateful and offensive content and the urgent need for models that are both multimodally and culturally nuanced. MultiHateClip represents a foundational advance in enhancing hateful video detection by underscoring the necessity of a multimodal and culturally sensitive approach in combating online hate speech.