mlsanityData to 14 Oct 2026

Sound Check: Auditing Audio Datasets

arXiv 2410.13114 ↗First posted 17 Oct 2024cs.SD cs.AI cs.CYcited by 3

Not on the map. Why?

Abstract (arXiv)

Generative audio models are rapidly advancing in both capabilities and public utilization -- several powerful generative audio models have readily available open weights, and some tech companies have released high quality generative audio products. Yet, while prior work has enumerated many ethical issues stemming from the data on which generative visual and textual models have been trained, we have little understanding of similar issues with generative audio datasets, including those related to bias, toxicity, and intellectual property. To bridge this gap, we conducted a literature review of hundreds of audio datasets and selected seven of the most prominent to audit in more detail. We found that these datasets are biased against women, contain toxic stereotypes about marginalized communities, and contain significant amounts of copyrighted work. To enable artists to see if they are in popular audio datasets and facilitate exploration of the contents of these datasets, we developed a web tool audio datasets exploration tool at https://audio-audit.vercel.app.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 68 of its 155 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

68 of 155 references.

References · 68

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2018Datasheets for Datasetscited by 1,386
  6. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  7. 2021Ethical and social risks of harm from Language Modelscited by 966
  8. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  9. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  10. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  11. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  12. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 599
38 more
  1. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  2. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
  3. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  4. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  5. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  6. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  7. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
  8. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  9. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  10. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 265
  11. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  12. 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 260
  13. 2018Gender Bias in Neural Natural Language Processingcited by 211
  14. 2016FMA: A Dataset For Music Analysiscited by 207
  15. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
  16. 2021The Values Encoded in Machine Learning Researchcited by 172
  17. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 156
  18. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 155
  19. 2022Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reductioncited by 138
  20. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 120
  21. 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 110
  22. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 108
  23. 2021Algorithmic Monoculture and Social Welfarecited by 100
  24. 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 91
  25. 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Modelscited by 83
  26. 2023Music ControlNet: Multiple Time-varying Controls for Music Generationcited by 83
  27. 2023The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluationcited by 68
  28. 2023ICASSP 2023 Deep Noise Suppression Challengecited by 65
  29. 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
  30. 2023VampNet: Music Generation via Masked Acoustic Token Modelingcited by 57
  31. 2023SingSong: Generating musical accompaniments from singingcited by 55
  32. 2024Careless Whisper: Speech-to-Text Hallucination Harmscited by 53
  33. 2023A Review of Deep Learning Techniques for Speech Processingcited by 52
  34. 2019Counterpoint by Convolutioncited by 43
  35. 2024AI auditing: The Broken Bus on the Road to AI Accountabilitycited by 43
  36. 2023Mass-Producing Failures of Multimodal Systems with Language Modelscited by 36
  37. 2023Queer In AI: A Case Study in Community-Led Participatory AIcited by 34
  38. 2023Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Riskscited by 31

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.