# Enhanced Multimodal Hate Video Detection via Channel-wise and Modality-wise Fusion

arXiv: https://arxiv.org/abs/2505.12051 · HTML: https://mlsanity.com/paper/2505.12051/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 17 May 2025.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [HateCheck: Functional Tests for Hate Speech Detection Models](https://mlsanity.com/paper/2012.15606/) (2020)
- [Deep Learning for Hate Speech Detection in Tweets](https://mlsanity.com/paper/1706.00188/) (2017)
- [Exploring Hate Speech Detection in Multimodal Publications](https://mlsanity.com/paper/1910.03814/) (2019)
- [Towards generalisable hate speech detection: a review on obstacles and solutions](https://mlsanity.com/paper/2102.08886/) (2021)
- [All You Need is "Love": Evading Hate-speech Detection](https://mlsanity.com/paper/1808.09115/) (2018)
- [HateMM: A Multi-Modal Dataset for Hate Video Classification](https://mlsanity.com/paper/2305.03915/) (2023)
- [A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines](https://mlsanity.com/paper/1801.01586/) (2018)
- [Understanding and Detecting Hateful Content using Contrastive Learning](https://mlsanity.com/paper/2201.08387/) (2022)
- [Modulated Fusion using Transformer for Linguistic-Acoustic Emotion Recognition](https://mlsanity.com/paper/2010.02057/) (2020)

## Built on by (most cited first, 8 of 8)

- [Revealing Temporal Label Noise in Multimodal Hateful Video Classification](https://mlsanity.com/paper/2508.04900/) (2025)
- [Multimodal Hate Detection Using Dual-Stream Graph Neural Networks](https://mlsanity.com/paper/2509.13515/) (2025)
- [MultiHateLoc: Towards Temporal Localisation of Multimodal Hate Content in Online Videos](https://mlsanity.com/paper/2512.10408/) (2025)
- [Reasoning-Aware Multimodal Fusion for Hateful Video Detection](https://mlsanity.com/paper/2512.02743/) (2025)
- [Text-Routed Sparse Mixture-of-Experts Model with Explanation and Temporal Alignment for Multi-Modal Sentiment Analysis](https://mlsanity.com/paper/2512.22741/) (2025)
- [Training-Free and Interpretable Hateful Video Detection via Multi-stage Adversarial Reasoning](https://mlsanity.com/paper/2601.15115/) (2026)
- [Towards Training-free Multimodal Hate Localisation with Large Language Models](https://mlsanity.com/paper/2602.09637/) (2026)
- [CLARA: Clip-Level Multimodal Alignment with VLM-Derived Rationales for Hateful Video Detection](https://mlsanity.com/paper/2608.15905/) (2026)

## Abstract (arXiv)

The rapid rise of video content on platforms such as TikTok and YouTube has transformed information dissemination, but it has also facilitated the spread of harmful content, particularly hate videos. Despite significant efforts to combat hate speech, detecting these videos remains challenging due to their often implicit nature. Current detection methods primarily rely on unimodal approaches, which inadequately capture the complementary features across different modalities. While multimodal techniques offer a broader perspective, many fail to effectively integrate temporal dynamics and modality-wise interactions essential for identifying nuanced hate content. In this paper, we present CMFusion, an enhanced multimodal hate video detection model utilizing a novel Channel-wise and Modality-wise Fusion Mechanism. CMFusion first extracts features from text, audio, and video modalities using pre-trained models and then incorporates a temporal cross-attention mechanism to capture dependencies between video and audio streams. The learned features are then processed by channel-wise and modality-wise fusion modules to obtain informative representations of videos. Our extensive experiments on a real-world dataset demonstrate that CMFusion significantly outperforms five widely used baselines in terms of accuracy, precision, recall, and F1 score. Comprehensive ablation studies and parameter analyses further validate our design choices, highlighting the model's effectiveness in detecting hate videos. The source codes will be made publicly available at https://github.com/EvelynZ10/cmfusion.