mlsanityData to 7 Oct 2026

Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation

arXiv 2407.05361 ↗First posted 7 Jul 2024eess.AS cs.CLcited by 200

Not on the map. Why?

Cited by 48 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advancements in speech generation models have been significantly driven by the use of large-scale training data. However, producing highly spontaneous, human-like speech remains a challenge due to the scarcity of large, diverse, and spontaneous speech datasets. In response, we introduce Emilia, the first large-scale, multilingual, and diverse speech generation dataset. Emilia starts with over 101k hours of speech across six languages, covering a wide range of speaking styles to enable more natural and spontaneous speech generation. To facilitate the scale-up of Emilia, we also present Emilia-Pipe, the first open-source preprocessing pipeline designed to efficiently transform raw, in-the-wild speech data into high-quality training data with speech annotations. Experimental results demonstrate the effectiveness of both Emilia and Emilia-Pipe. Demos are available at: https://emilia-dataset.github.io/Emilia-Demo-Page/.

Key facts. Cited by 200 papers in the map (data through 7 Oct 2026). Builds on 22 of its 26 references that are in the map. First posted on arXiv 7 Jul 2024.

Builds on

22 of 26 references.

References · 22

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  4. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  5. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  6. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  7. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  8. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  9. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  10. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  11. 2023WhisperX: Time-Accurate Speech Transcription of Long-Form Audiocited by 320
  12. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
10 more
  1. 2021A Survey on Neural Speech Synthesiscited by 182
  2. 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 165
  3. 2023SoundStorm: Efficient Parallel Audio Generationcited by 111
  4. 2023Powerset multi-class cross entropy loss for neural speaker diarizationcited by 109
  5. 2024WenetSpeech4TTS: A 12,800-hour Mandarin TTS Corpus for Large Speech Generation Model Benchmarkcited by 39
  6. 2023Amphion: An Open-Source Audio, Music and Speech Generation Toolkitcited by 25
  7. 2021KUIELab-MDX-Net: A Two-Stream Neural Network for Music Demixingcited by 24
  8. 2023AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Datacited by 16
  9. 2020How Far Are We from Robust Voice Conversion: A Surveycited by 14
  10. 2023Multi-Scale Sub-Band Constant-Q Transform Discriminator for High-Fidelity Vocodercited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 102; 2026: 84.

200 papers.

Sort:

By area:

  1. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  2. 2025Kimi-Audio Technical Reportcited by 210
  3. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  4. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 109
  5. 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmarkcited by 101
  6. 2024WavChat: A Survey of Spoken Dialogue Modelscited by 88
  7. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 85
  8. 2025MiMo-Audio: Audio Language Models are Few-Shot Learnerscited by 58
  9. 2025DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generationcited by 56
  10. 2025OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLMcited by 53
  11. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 47
  12. 2024Zero-shot Voice Conversion with Diffusion Transformerscited by 39
38 more
  1. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 38
  2. 2025UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactionscited by 35
  3. 2025VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Modelcited by 30
  4. 2025ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matchingcited by 30
  5. 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialoguecited by 27
  6. 2025Streaming Sequence-to-Sequence Learning with Delayed Streams Modelingcited by 24
  7. 2025AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancementcited by 23
  8. 2025XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecscited by 21
  9. 2025Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generationcited by 19
  10. 2025MoonCast: High-Quality Zero-Shot Podcast Generationcited by 18
  11. 2025Scaling Rich Style-Prompted Text-to-Speech Datasetscited by 17
  12. 2025TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modelingcited by 15
  13. 2025Ming-UniAudio: Speech LLM for Joint Understanding, Generation and Editing with Unified Representationcited by 15
  14. 2026Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Musiccited by 15
  15. 2025ContextASR-Bench: A Massive Contextual Speech Recognition Benchmarkcited by 14
  16. 2025NonverbalTTS: A Public English Corpus of Text-Aligned Nonverbal Vocalizations with Emotion Annotations for Text-to-Speechcited by 14
  17. 2025Closing the Gap Between Text and Speech Understanding in LLMscited by 14
  18. 2025TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modelingcited by 12
  19. 2025CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speechcited by 11
  20. 2026SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing modelcited by 11
  21. 2024Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruningcited by 10
  22. 2024TouchTTS: An Embarrassingly Simple TTS Framework that Everyone Can Touchcited by 10
  23. 2025Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech Synthesiscited by 10
  24. 2025ESPnet-SpeechLM: An Open Speech Language Model Toolkitcited by 9
  25. 2025OpusLM: A Family of Open Unified Speech Language Modelscited by 9
  26. 2026CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillationcited by 9
  27. 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusioncited by 9
  28. 2025OpenS2S: Advancing Fully Open-Source End-to-End Empathetic Large Speech Language Modelcited by 8
  29. 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systemscited by 7
  30. 2025ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matchingcited by 7
  31. 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesiscited by 7
  32. 2025Multi-Metric Preference Alignment for Generative Speech Restorationcited by 7
  33. 2025MGM-Omni: Scaling Omni LLMs to Personalized Long-Horizon Speechcited by 7
  34. 2024Text-To-Speech Synthesis In The Wildcited by 6
  35. 2025MNV-17: A High-Quality Performative Mandarin Dataset for Nonverbal Vocalization Recognition in Speechcited by 6
  36. 2025UniSS: Unified Expressive Speech-to-Speech Translation with Your Voicecited by 6
  37. 2025M3-TTS: Multi-modal DiT Alignment & Mel-latent for Zero-shot High-fidelity Speech Synthesiscited by 6
  38. 2024KALL-E:Autoregressive Speech Synthesis with Next-Distribution Predictioncited by 5

50 most cited of 200 All 200 (JSON)