Sound Check: Auditing Audio Datasets
Not on the map. Why?
Abstract (arXiv)
Generative audio models are rapidly advancing in both capabilities and public utilization -- several powerful generative audio models have readily available open weights, and some tech companies have released high quality generative audio products. Yet, while prior work has enumerated many ethical issues stemming from the data on which generative visual and textual models have been trained, we have little understanding of similar issues with generative audio datasets, including those related to bias, toxicity, and intellectual property. To bridge this gap, we conducted a literature review of hundreds of audio datasets and selected seven of the most prominent to audit in more detail. We found that these datasets are biased against women, contain toxic stereotypes about marginalized communities, and contain significant amounts of copyrighted work. To enable artists to see if they are in popular audio datasets and facilitate exploration of the contents of these datasets, we developed a web tool audio datasets exploration tool at https://audio-audit.vercel.app.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 68 of its 155 references that are in the map. First posted on arXiv 17 Oct 2024.
Builds on
68 of 155 references.
References · 68
- 2017Attention Is All You Needcited by 60,361
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2018Datasheets for Datasetscited by 1,386
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
- 2023DataComp: In search of the next generation of multimodal datasetscited by 644
- 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
- 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
- 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 599
38 more
- 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
- 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
- 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
- 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
- 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
- 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
- 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
- 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
- 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
- 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 265
- 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
- 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 260
- 2018Gender Bias in Neural Natural Language Processingcited by 211
- 2016FMA: A Dataset For Music Analysiscited by 207
- 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
- 2021The Values Encoded in Machine Learning Researchcited by 172
- 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 156
- 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 155
- 2022Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reductioncited by 138
- 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 120
- 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 110
- 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 108
- 2021Algorithmic Monoculture and Social Welfarecited by 100
- 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 91
- 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Modelscited by 83
- 2023Music ControlNet: Multiple Time-varying Controls for Music Generationcited by 83
- 2023The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluationcited by 68
- 2023ICASSP 2023 Deep Noise Suppression Challengecited by 65
- 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
- 2023VampNet: Music Generation via Masked Acoustic Token Modelingcited by 57
- 2023SingSong: Generating musical accompaniments from singingcited by 55
- 2024Careless Whisper: Speech-to-Text Hallucination Harmscited by 53
- 2023A Review of Deep Learning Techniques for Speech Processingcited by 52
- 2019Counterpoint by Convolutioncited by 43
- 2024AI auditing: The Broken Bus on the Road to AI Accountabilitycited by 43
- 2023Mass-Producing Failures of Multimodal Systems with Language Modelscited by 36
- 2023Queer In AI: A Case Study in Community-Led Participatory AIcited by 34
- 2023Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Riskscited by 31
50 most cited of 68 All 68 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.
- 2025XAttnMark: Learning Robust Audio Watermarking with Cross-Attentioncited by 5
- 2026Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarkingcited by 2
- 2026V.O.I.C.E (Voice, Ownership, Identity, Control, Expression): Risk Taxonomy of Synthetic Voice Generation From Empirical Datacited by 0
- 2026V.O.I.C.E (Voice, Ownership, Identity, Control, Expression): Risk Taxonomy of Synthetic Voice Generation From Empirical Data25 Apr 2026
- 2026Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarking15 Apr 2026
- 2025XAttnMark: Learning Robust Audio Watermarking with Cross-Attention6 Feb 2025