mlsanityData to 7 Oct 2026

LibriheavyMix: A 20,000-Hour Dataset for Single-Channel Reverberant Multi-Talker Speech Separation, ASR and Speaker Diarization

arXiv 2409.00819 ↗First posted 1 Sep 2024cs.SD cs.CL eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

The evolving speech processing landscape is increasingly focused on complex scenarios like meetings or cocktail parties with multiple simultaneous speakers and far-field conditions. Existing methodologies for addressing these challenges fall into two categories: multi-channel and single-channel solutions. Single-channel approaches, notable for their generality and convenience, do not require specific information about microphone arrays. This paper presents a large-scale far-field overlapping speech dataset, crafted to advance research in speech separation, recognition, and speaker diarization. This dataset is a critical resource for decoding ``Who said What and When'' in multi-talker, reverberant environments, a daunting challenge in the field. Additionally, we introduce a pipeline system encompassing speech separation, recognition, and diarization as a foundational benchmark. Evaluations on the WHAMR! dataset validate the broad applicability of the proposed data.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 38 of its 43 references that are in the map. First posted on arXiv 1 Sep 2024.

Builds on

38 of 43 references.

References · 38

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  4. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  5. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
  6. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 364
  7. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 211
  8. 2016Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separationcited by 189
  9. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 154
  10. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 154
  11. 2020CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordingscited by 129
  12. 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 122
26 more
  1. 2023Powerset multi-class cross entropy loss for neural speaker diarizationcited by 109
  2. 2020Serialized Output Training for End-to-End Overlapped Speech Recognitioncited by 87
  3. 2019End-to-End Neural Speaker Diarization with Permutation-Free Objectivescited by 86
  4. 2019Transformer-based Acoustic Modeling for Hybrid Speech Recognitioncited by 86
  5. 2023Libriheavy: a 50,000 hours ASR corpus with punctuation casing and contextcited by 84
  6. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 77
  7. 2020Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenariocited by 67
  8. 2021M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challengecited by 67
  9. 2019WHAMR!: Noisy and Reverberant Single-Channel Speech Separationcited by 61
  10. 2019MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognitioncited by 44
  11. 2023MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separationcited by 39
  12. 2022Streaming Multi-Talker ASR with Token-Level Serialized Output Trainingcited by 37
  13. 2021End-to-End Speaker-Attributed ASR with Transformercited by 35
  14. 2021FAST-RIR: Fast neural diffuse room impulse response generatorcited by 35
  15. 2017Recognizing Multi-talker Speech with Permutation Invariant Trainingcited by 33
  16. 2021Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphonecited by 33
  17. 2022From Simulated Mixtures to Simulated Conversations as Training Data for End-to-End Neural Diarizationcited by 20
  18. 2021Target-speaker Voice Activity Detection with Improved I-Vector Estimation for Unknown Number of Speakercited by 15
  19. 2020An empirical study of Conv-TasNetcited by 14
  20. 2023A Sidecar Separator Can Convert a Single-Talker Speech Recognition System to a Multi-Talker Onecited by 13
  21. 2023Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separatorcited by 7
  22. 2023Semantic VAD: Low-Latency Voice Activity Detection for Speech Interactioncited by 6
  23. 2023CASA-ASR: Context-Aware Speaker-Attributed ASRcited by 6
  24. 2022A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetingscited by 5
  25. 2022Unified End-to-End Speech Recognition and Endpointing for Fast and Efficient Speech Systemscited by 5
  26. 2022A Comparative Study on Multichannel Speaker-Attributed Automatic Speech Recognition in Multi-party Meetingscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audiocited by 4
  2. 2026Speaker-Aware Simulation Improves Conversational Speech Recognitioncited by 2
  3. 2025Towards Robust Overlapping Speech Detection: A Speaker-Aware Progressive Approach Using WavLMcited by 1