# YODAS: Youtube-Oriented Dataset for Audio and Speech

arXiv: https://arxiv.org/abs/2406.00899 · HTML: https://mlsanity.com/paper/2406.00899/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 2 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages](https://mlsanity.com/paper/2303.01037/) (2023)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [VoxLingua107: a Dataset for Spoken Language Recognition](https://mlsanity.com/paper/2011.12998/) (2020)
- [Scaling Laws for Neural Machine Translation](https://mlsanity.com/paper/2109.07740/) (2021)
- [Universal Phone Recognition with a Multilingual Allophone System](https://mlsanity.com/paper/2002.11800/) (2020)
- [CTC-Segmentation of Large Corpora for German End-to-end Speech Recognition](https://mlsanity.com/paper/2007.09127/) (2020)
- [Wav2Letter: an End-to-End ConvNet-based Speech Recognition System](https://mlsanity.com/paper/1609.03193/) (2016)
- [JTubeSpeech: corpus of Japanese speech collected from YouTube for speech recognition and speaker verification](https://mlsanity.com/paper/2112.09323/) (2021)

## Built on by (most cited first, 30 of 60)

- [Towards Robust Speech Representation Learning for Thousands of Languages](https://mlsanity.com/paper/2407.00837/) (2024)
- [Consent in Crisis: The Rapid Decline of the AI Data Commons](https://mlsanity.com/paper/2407.14933/) (2024)
- [Distilling an End-to-End Voice Assistant Without Instruction Training Data](https://mlsanity.com/paper/2410.02678/) (2024)
- [GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement](https://mlsanity.com/paper/2406.11546/) (2024)
- [OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning](https://mlsanity.com/paper/2506.00338/) (2025)
- [Granary: Speech Recognition and Translation Dataset in 25 European Languages](https://mlsanity.com/paper/2505.13404/) (2025)
- [Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation](https://mlsanity.com/paper/2501.15907/) (2025)
- [Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead](https://mlsanity.com/paper/2505.21315/) (2025)
- [CS-FLEURS: A Massively Multilingual and Code-Switched Speech Dataset](https://mlsanity.com/paper/2509.14161/) (2025)
- [WAXAL: A Large-Scale Multilingual African Language Speech Corpus](https://mlsanity.com/paper/2602.02734/) (2026)
- [Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models](https://mlsanity.com/paper/2409.10999/) (2024)
- [GLAP: General contrastive audio-text pretraining across domains and languages](https://mlsanity.com/paper/2506.11350/) (2025)
- [ESPnet-SpeechLM: An Open Speech Language Model Toolkit](https://mlsanity.com/paper/2502.15218/) (2025)
- [Loquacious Set: 25,000 Hours of Transcribed and Diverse English Speech Recognition Data for Research and Commercial Use](https://mlsanity.com/paper/2505.21578/) (2025)
- [OpusLM: A Family of Open Unified Speech Language Models](https://mlsanity.com/paper/2506.17611/) (2025)
- [Dolphin: A Large-Scale Automatic Speech Recognition Model for Eastern Languages](https://mlsanity.com/paper/2503.20212/) (2025)
- [SeaLLMs-Audio: Large Audio-Language Models for Southeast Asia](https://mlsanity.com/paper/2511.01670/) (2025)
- [The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning](https://mlsanity.com/paper/2603.17837/) (2026)
- [OLMoASR: Open Models and Data for Training Robust Speech Recognition Models](https://mlsanity.com/paper/2508.20869/) (2025)
- [StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMs](https://mlsanity.com/paper/2509.22220/) (2025)
- [EuroSpeech: A Multilingual Speech Corpus](https://mlsanity.com/paper/2510.00514/) (2025)
- [Scaling Open Discrete Audio Foundation Models with Interleaved Semantic, Acoustic, and Text Tokens](https://mlsanity.com/paper/2602.16687/) (2026)
- [Whale: Large-Scale multilingual ASR model with w2v-BERT and E-Branchformer with large speech data](https://mlsanity.com/paper/2506.01439/) (2025)
- [DuplexChat: Constructing Speaker-Separated Full-Duplex Dialogue Speech at Scale for Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2607.04941/) (2026)
- [Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioning](https://mlsanity.com/paper/2506.04527/) (2025)
- [CAM~OES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese](https://mlsanity.com/paper/2508.19721/) (2025)
- [Thai Semantic End-of-Turn Detection for Real-Time Voice Agents](https://mlsanity.com/paper/2510.04016/) (2025)
- [Extending Audio Context for Long-Form Understanding in Large Audio-Language Models](https://mlsanity.com/paper/2510.15231/) (2025)
- [NaturalVoices: A Large-Scale, Spontaneous and Emotional Podcast Dataset for Voice Conversion](https://mlsanity.com/paper/2511.00256/) (2025)
- [Enhancing Large Language Model-based Speech Recognition by Contextualization for Rare and Ambiguous Words](https://mlsanity.com/paper/2408.08027/) (2024)

## Abstract (arXiv)

In this study, we introduce YODAS (YouTube-Oriented Dataset for Audio and Speech), a large-scale, multilingual dataset comprising currently over 500k hours of speech data in more than 100 languages, sourced from both labeled and unlabeled YouTube speech datasets. The labeled subsets, including manual or automatic subtitles, facilitate supervised model training. Conversely, the unlabeled subsets are apt for self-supervised learning applications. YODAS is distinctive as the first publicly available dataset of its scale, and it is distributed under a Creative Commons license. We introduce the collection methodology utilized for YODAS, which contributes to the large-scale speech dataset construction. Subsequently, we provide a comprehensive analysis of speech, text contained within the dataset. Finally, we describe the speech recognition baselines over the top-15 languages.