mlsanityData to 14 Oct 2026

Sommelier: Scalable Open Multi-turn Audio Pre-processing for Full-duplex Speech Language Models

arXiv 2603.25750 ↗First posted 20 Mar 2026cs.SD cs.AI eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

As the paradigm of AI shifts from text-based LLMs to Speech Language Models (SLMs), there is a growing demand for full-duplex systems capable of real-time, natural human-computer interaction. However, the development of such models is constrained by the scarcity of high-quality, multi-speaker conversational data, as existing large-scale resources are predominantly single-speaker or limited in volume. Addressing the complex dynamics of natural dialogue, such as overlapping and back-channeling remains a challenge, with standard processing pipelines suffering from diarization errors and ASR hallucinations. To bridge this gap, we present a robust and scalable open-source data processing pipeline designed for full-duplex model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 60 references that are in the map. First posted on arXiv 20 Mar 2026.

Builds on

39 of 60 references.

References · 39

  1. 2024GPT-4o System Cardcited by 4,588
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  4. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  5. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
  6. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  7. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
  8. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  9. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  10. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  11. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 226
  12. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
27 more
  1. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  2. 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 182
  3. 2022Hybrid Transformers for Music Source Separationcited by 134
  4. 2023Powerset multi-class cross entropy loss for neural speaker diarizationcited by 111
  5. 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 100
  6. 2021Hybrid Spectrogram and Waveform Source Separationcited by 91
  7. 2020Spot the conversation: speaker diarisation in the wildcited by 79
  8. 2026PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Modelscited by 63
  9. 2025Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and ASTcited by 60
  10. 2025OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLMcited by 59
  11. 2023Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognitioncited by 56
  12. 2025Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Modelscited by 54
  13. 2024Careless Whisper: Speech-to-Text Hallucination Harmscited by 53
  14. 2024A Full-duplex Speech Dialogue Scheme Based On Large Language Modelscited by 40
  15. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  16. 2024DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factorscited by 32
  17. 2025Investigation of Whisper ASR Hallucinations Induced by Non-Speech Audiocited by 31
  18. 2025SAM Audio: Segment Anything in Audiocited by 30
  19. 2025Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generationcited by 22
  20. 2024Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusioncited by 20
  21. 2024Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systemscited by 15
  22. 2024Separate and Reconstruct: Asymmetric Encoder-Decoder for Speech Separationcited by 14
  23. 2025CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speechcited by 11
  24. 2025SpeechWeave: Diverse Multilingual Synthetic Text & Audio Data Generation Pipeline for Training Text to Speech Modelscited by 5
  25. 2024Benchmarking Japanese Speech Recognition on ASR-LLM Setups with Multi-Pass Augmented Generative Error Correctioncited by 4
  26. 2024TeXBLEU: Automatic Metric for Evaluate LaTeX Formatcited by 3
  27. 2025Data-Centric Lessons To Improve Speech-Language Pretrainingcited by 3

Built on by

Not cited yet.