Unveiling Covert Toxicity in Multimodal Data via Toxicity Association Graphs: A Graph-Based Metric and Interpretable Detection Framework
Not on the map. Why?
Abstract (arXiv)
Detecting toxicity in multimodal data remains a significant challenge, as harmful meanings often lurk beneath seemingly benign individual modalities: only emerging when modalities are combined and semantic associations are activated. To address this, we propose a novel detection framework based on Toxicity Association Graphs (TAGs), which systematically model semantic associations between innocuous entities and latent toxic implications. Leveraging TAGs, we introduce the first quantifiable metric for hidden toxicity, the Multimodal Toxicity Covertness (MTC), which measures the degree of concealment in toxic multimodal expressions. By integrating our detection framework with the MTC metric, our approach enables precise identification of covert toxicity while preserving full interpretability of the decision-making process, significantly enhancing transparency in multimodal toxicity detection. To validate our method, we construct the Covert Toxic Dataset, the first benchmark specifically designed to capture high-covertness toxic multimodal instances. This dataset encodes nuanced cross-modal associations and serves as a rigorous testbed for evaluating both the proposed metric and detection framework. Extensive experiments demonstrate that our approach outperforms existing methods across both low- and high-covertness toxicity regimes, while delivering clear, interpretable, and auditable detection outcomes. Together, our contributions advance the state of the art in explainable multimodal toxicity detection and lay the foundation for future context-aware and interpretable approaches. Content Warning: This paper contains examples of toxic multimodal content that may be offensive or disturbing to some readers. Reader discretion is advised.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 3 Feb 2026.
Builds on
25 of 44 references.
References · 25
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
- 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 85
- 2021Detecting Harmful Memes and Their Targetscited by 77
- 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 59
- 2022Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Featurescited by 55
- 2020Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challengecited by 51
- 2020Vilio: State-of-the-art Visio-Linguistic Models applied to Hateful Memescited by 50
- 2024GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abusecited by 50
- 2024ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectorscited by 48
13 more
- 2021Detecting Propaganda Techniques in Memescited by 43
- 2024MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classificationcited by 29
- 2022DISARM: Detecting the Victims Targeted by Harmful Memescited by 20
- 2022DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformationcited by 13
- 2024Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanationscited by 13
- 2024Towards Comprehensive Detection of Chinese Harmful Memescited by 12
- 2024ToxVidLM: A Multimodal Framework for Toxicity Detection in Code-Mixed Videoscited by 10
- 2025Efficient LLaMA-3.2-Vision by Trimming Cross-attended Visual Featurescited by 10
- 2021Benchmark dataset of memes with text transcriptions for automatic detection of multi-modal misogynistic contentcited by 9
- 2023A Template Is All You Memecited by 9
- 2025ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMscited by 5
- 2022MemeTector: Enforcing deep focus for meme detectioncited by 4
- 2024Text or Image? What is More Important in Cross-Domain Generalization Capabilities of Hate Meme Detection Models?cited by 4
Built on by
Not cited yet.