# DeToxy: A Large-Scale Multimodal Dataset for Toxicity Classification in Spoken Utterances

arXiv: https://arxiv.org/abs/2110.07592 · HTML: https://mlsanity.com/paper/2110.07592/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation](https://mlsanity.com/paper/1805.04699/) (2018)
- [Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders](https://mlsanity.com/paper/1910.12638/) (2019)
- [MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos](https://mlsanity.com/paper/1606.06259/) (2016)
- [Deep Learning for Hate Speech Detection in Tweets](https://mlsanity.com/paper/1706.00188/) (2017)
- [Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training](https://mlsanity.com/paper/2104.01027/) (2021)
- [DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantization](https://mlsanity.com/paper/2012.06659/) (2020)
- [What all do audio transformer models hear? Probing Acoustic Representations for Language Delivery and its Structure](https://mlsanity.com/paper/2101.00387/) (2021)
- [Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis](https://mlsanity.com/paper/2010.04543/) (2020)
- [End-to-end Named Entity Recognition from English Speech](https://mlsanity.com/paper/2005.11184/) (2020)
- [MaSS: A Large and Clean Multilingual Corpus of Sentence-aligned Spoken Utterances Extracted from the Bible](https://mlsanity.com/paper/1907.12895/) (2019)
- [Speech Sentiment Analysis via Pre-trained Features from End-to-end ASR Models](https://mlsanity.com/paper/1911.09762/) (2019)
- [Audio De-identification: A New Entity Recognition Task](https://mlsanity.com/paper/1903.07037/) (2019)
- [audino: A Modern Annotation Tool for Audio and Speech](https://mlsanity.com/paper/2006.05236/) (2020)
- [Incorporating End-to-End Speech Recognition Models for Sentiment Analysis](https://mlsanity.com/paper/1902.11245/) (2019)

## Built on by (most cited first, 14 of 14)

- [Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models](https://mlsanity.com/paper/2401.16727/) (2024)
- [MuTox: Universal MUltilingual Audio-based TOXicity Dataset and Zero-shot Detector](https://mlsanity.com/paper/2401.05060/) (2024)
- [Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities](https://mlsanity.com/paper/2406.07353/) (2024)
- [Lightweight Toxicity Detection in Spoken Language: A Transformer-based Approach for Edge Devices](https://mlsanity.com/paper/2304.11408/) (2023)
- [Added Toxicity Mitigation at Inference Time for Multimodal and Massively Multilingual Translation](https://mlsanity.com/paper/2311.06532/) (2023)
- [Enhancing Multilingual Voice Toxicity Detection with Speech-Text Alignment](https://mlsanity.com/paper/2406.10325/) (2024)
- [Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insights](https://mlsanity.com/paper/2406.17430/) (2024)
- [ToxicTone: A Mandarin Audio Dataset Annotated for Toxicity and Toxic Utterance Tonality](https://mlsanity.com/paper/2505.15773/) (2025)
- [Attentive Fusion: A Transformer-based Approach to Multimodal Hate Speech Detection](https://mlsanity.com/paper/2401.10653/) (2024)
- [On the Role of Speech Data in Reducing Toxicity Detection Bias](https://mlsanity.com/paper/2411.08135/) (2024)
- [Towards Cross-Lingual Audio Abuse Detection in Low-Resource Settings with Few-Shot Learning](https://mlsanity.com/paper/2412.01408/) (2024)
- [ViToSA: Audio-Based Toxic Spans Detection on Vietnamese Speech Utterances](https://mlsanity.com/paper/2506.00636/) (2025)
- ["bot lane noob" Towards Deployment of NLP-based Toxicity Detectors in Video Games](https://mlsanity.com/paper/2604.10175/) (2026)
- [Beyond Content: A Comprehensive Speech Toxicity Dataset and Detection Framework Incorporating Paralinguistic Cues](https://mlsanity.com/paper/2605.15984/) (2026)

## Abstract (arXiv)

Toxic speech, also known as hate speech, is regarded as one of the crucial issues plaguing online social media today. Most recent work on toxic speech detection is constrained to the modality of text and written conversations with very limited work on toxicity detection from spoken utterances or using the modality of speech. In this paper, we introduce a new dataset DeToxy, the first publicly available toxicity annotated dataset for the English language. DeToxy is sourced from various openly available speech databases and consists of over 2 million utterances. We believe that our dataset would act as a benchmark for the relatively new and un-explored Spoken Language Processing task of detecting toxicity from spoken utterances and boost further research in this space. Finally, we also provide strong unimodal baselines for our dataset and compare traditional two-step and E2E approaches. Our experiments show that in the case of spoken utterances, text-based approaches are largely dependent on gold human-annotated transcripts for their performance and also suffer from the problem of keyword bias. However, the presence of speech files in DeToxy helps facilitates the development of E2E speech models which alleviate both the above-stated problems by better capturing speech clues.