# HateMM: A Multi-Modal Dataset for Hate Video Classification

arXiv: https://arxiv.org/abs/2305.03915 · HTML: https://mlsanity.com/paper/2305.03915/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 6 May 2023.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [Deep Learning for Hate Speech Detection in Tweets](https://mlsanity.com/paper/1706.00188/) (2017)
- [Exploring Hate Speech Detection in Multimodal Publications](https://mlsanity.com/paper/1910.03814/) (2019)
- [Contextualizing Hate Speech Classifiers with Post-hoc Explanation](https://mlsanity.com/paper/2005.02439/) (2020)
- [Empath: Understanding Topic Signals in Large-Scale Text](https://mlsanity.com/paper/1602.06979/) (2016)
- [Detecting Hate Speech in Multi-modal Memes](https://mlsanity.com/paper/2012.14891/) (2020)
- [A Unified Deep Learning Architecture for Abuse Detection](https://mlsanity.com/paper/1802.00385/) (2018)
- [Data Bootstrapping Approaches to Improve Low Resource Abusive Language Detection for Indic Languages](https://mlsanity.com/paper/2204.12543/) (2022)
- [Emotion Based Hate Speech Detection using Multimodal Learning](https://mlsanity.com/paper/2202.06218/) (2022)
- [Audio style transfer](https://mlsanity.com/paper/1710.11385/) (2017)
- [A little goes a long way: Improving toxic language classification despite data scarcity](https://mlsanity.com/paper/2009.12344/) (2020)
- [Unsupervised Cyberbullying Detection via Time-Informed Gaussian Mixture Model](https://mlsanity.com/paper/2008.02642/) (2020)
- [What is BitChute? Characterizing the "Free Speech" Alternative to YouTube](https://mlsanity.com/paper/2004.01984/) (2020)

## Built on by (most cited first, 30 of 42)

- [Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models](https://mlsanity.com/paper/2401.16727/) (2024)
- [MultiHateClip: A Multilingual Benchmark Dataset for Hateful Video Detection on YouTube and Bilibili](https://mlsanity.com/paper/2408.03468/) (2024)
- [MetaHate: A Dataset for Unifying Efforts on Hate Speech Detection](https://mlsanity.com/paper/2401.06526/) (2024)
- [ImpliHateVid: A Benchmark Dataset and Two-stage Contrastive Learning Framework for Implicit Hate Speech Detection in Videos](https://mlsanity.com/paper/2508.06570/) (2025)
- [ToxVidLM: A Multimodal Framework for Toxicity Detection in Code-Mixed Videos](https://mlsanity.com/paper/2405.20628/) (2024)
- [Cross-Modal Transfer from Memes to Videos: Addressing Data Scarcity in Hateful Video Detection](https://mlsanity.com/paper/2501.15438/) (2025)
- [Towards a Robust Framework for Multimodal Hate Detection: A Study on Video vs. Image-based Content](https://mlsanity.com/paper/2502.07138/) (2025)
- [Enhanced Multimodal Hate Video Detection via Channel-wise and Modality-wise Fusion](https://mlsanity.com/paper/2505.12051/) (2025)
- [Decoding Hate: Exploring Language Models' Reactions to Hate Speech](https://mlsanity.com/paper/2410.00775/) (2024)
- [Towards Efficient and Explainable Hate Speech Detection via Model Distillation](https://mlsanity.com/paper/2412.13698/) (2024)
- [MM-HSD: Multi-Modal Hate Speech Detection in Videos](https://mlsanity.com/paper/2508.20546/) (2025)
- [CPFD: Confidence-aware Privileged Feature Distillation for Short Video Classification](https://mlsanity.com/paper/2410.03038/) (2024)
- [Embedding-based Retrieval in Multimodal Content Moderation](https://mlsanity.com/paper/2507.01066/) (2025)
- [HateClipSeg: A Segment-Level Annotated Dataset for Fine-Grained Hate Video Detection](https://mlsanity.com/paper/2508.01712/) (2025)
- [Revealing Temporal Label Noise in Multimodal Hateful Video Classification](https://mlsanity.com/paper/2508.04900/) (2025)
- [OutSafe-Bench: A Benchmark for Multimodal Offensive Content Detection in Large Language Models](https://mlsanity.com/paper/2511.10287/) (2025)
- [Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: a Case Study on Hateful Memes](https://mlsanity.com/paper/2308.11585/) (2023)
- [Overview of the HASOC Subtrack at FIRE 2023: Identification of Tokens Contributing to Explicit Hate in English by Span Detection](https://mlsanity.com/paper/2311.09834/) (2023)
- [Towards Patronizing and Condescending Language in Chinese Videos: A Multimodal Dataset and Detector](https://mlsanity.com/paper/2409.05005/) (2024)
- [MuSeD: A Multimodal Spanish Dataset for Sexism Detection in Social Media Videos](https://mlsanity.com/paper/2504.11169/) (2025)
- [Multimodal Hate Detection Using Dual-Stream Graph Neural Networks](https://mlsanity.com/paper/2509.13515/) (2025)
- [Bridging Gaps in Hate Speech Detection: Meta-Collections and Benchmarks for Low-Resource Iberian Languages](https://mlsanity.com/paper/2510.11167/) (2025)
- [StreamSense: Streaming Social Task Detection with Selective Vision-Language Model Routing](https://mlsanity.com/paper/2601.22738/) (2026)
- [MTikGuard System: A Transformer-Based Multimodal System for Child-Safe Content Moderation on TikTok](https://mlsanity.com/paper/2511.17955/) (2025)
- [Can LLMs Evaluate What They Cannot Annotate? Revisiting LLM Reliability in Hate Speech Detection](https://mlsanity.com/paper/2512.09662/) (2025)
- [MultiHateLoc: Towards Temporal Localisation of Multimodal Hate Content in Online Videos](https://mlsanity.com/paper/2512.10408/) (2025)
- [Shedding the Facades, Connecting the Domains: Detecting Shifting Multimodal Hate Video with Test-Time Adaptation](https://mlsanity.com/paper/2602.00132/) (2026)
- [Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization](https://mlsanity.com/paper/2609.10410/) (2026)
- [Safe Guard: an LLM-agent for Real-time Voice-based Hate Speech Detection in Social Virtual Reality](https://mlsanity.com/paper/2409.15623/) (2024)
- [Personalisation or Prejudice? Addressing Geographic Bias in Hate Speech Detection using Debias Tuning in Large Language Models](https://mlsanity.com/paper/2505.02252/) (2025)

## Abstract (arXiv)

Hate speech has become one of the most significant issues in modern society, having implications in both the online and the offline world. Due to this, hate speech research has recently gained a lot of traction. However, most of the work has primarily focused on text media with relatively little work on images and even lesser on videos. Thus, early stage automated video moderation techniques are needed to handle the videos that are being uploaded to keep the platform safe and healthy. With a view to detect and remove hateful content from the video sharing platforms, our work focuses on hate video detection using multi-modalities. To this end, we curate ~43 hours of videos from BitChute and manually annotate them as hate or non-hate, along with the frame spans which could explain the labelling decision. To collect the relevant videos we harnessed search keywords from hate lexicons. We observe various cues in images and audio of hateful videos. Further, we build deep learning multi-modal models to classify the hate videos and observe that using all the modalities of the videos improves the overall hate speech detection performance (accuracy=0.798, macro F1-score=0.790) by ~5.7% compared to the best uni-modal model in terms of macro F1 score. In summary, our work takes the first step toward understanding and modeling hateful videos on video hosting platforms such as BitChute.