mlsanityData to 7 Oct 2026

Towards Unsupervised Speech Recognition at the Syllable-Level

arXiv 2510.03639 ↗First posted 4 Oct 2025cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Training speech recognizers with unpaired speech and text -- known as unsupervised speech recognition (UASR) -- is a crucial step toward extending ASR to low-resource languages in the long-tail distribution and enabling multimodal learning from non-parallel data. However, existing approaches based on phones often rely on costly resources such as grapheme-to-phoneme converters (G2Ps) and struggle to generalize to languages with ambiguous phoneme boundaries due to training instability. In this paper, we address both challenges by introducing a syllable-level UASR framework based on masked language modeling, which avoids the need for G2P and the instability of GAN-based methods. Our approach achieves up to a 40% relative reduction in character error rate (CER) on LibriSpeech and generalizes effectively to Mandarin, a language that has remained particularly difficult for prior methods. Code will be released upon acceptance.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 41 of its 47 references that are in the map. First posted on arXiv 4 Oct 2025.

Builds on

41 of 47 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  6. 2024Qwen2-Audio Technical Reportcited by 543
  7. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  8. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  9. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  10. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 340
  11. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  12. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
29 more
  1. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 177
  2. 2017Unsupervised Neural Machine Translationcited by 148
  3. 2021Unsupervised Speech Recognitioncited by 147
  4. 2018Phrase-Based & Neural Unsupervised Machine Translationcited by 138
  5. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
  6. 2025SuperBPE: Space Travel for Language Modelscited by 58
  7. 2018Unsupervised Statistical Machine Translationcited by 51
  8. 2020Text-Free Image-to-Speech Synthesis Using Learned Segmental Unitscited by 50
  9. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 48
  10. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 44
  11. 2019An Effective Approach to Unsupervised Machine Translationcited by 37
  12. 2022Word Discovery in Visually Grounded, Self-Supervised Speech Modelscited by 37
  13. 2024SyllableLM: Learning Coarse Semantic Units for Speech Language Modelscited by 24
  14. 2018Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddingscited by 22
  15. 2018Non-Adversarial Unsupervised Word Translationcited by 19
  16. 2024Sylber: Syllabic Embedding Representation of Speech from Raw Audiocited by 19
  17. 2019Unpaired Image-to-Speech Synthesis with Multimodal Information Bottleneckcited by 15
  18. 2021Unsupervised Speech Segmentation and Variable Rate Representation Learning using Segmental Contrastive Predictive Codingcited by 14
  19. 2022Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognitioncited by 14
  20. 2023SD-HuBERT: Sentence-Level Self-Distillation Induces Syllabic Organization in HuBERTcited by 13
  21. 2022Simple and Effective Unsupervised Speech Synthesiscited by 12
  22. 2022Simple and Effective Unsupervised Speech Translationcited by 12
  23. 2019Completely Unsupervised Speech Recognition By A Generative Adversarial Network Harmonized With Iteratively Refined Hidden Markov Modelscited by 8
  24. 2023Unsupervised Word Segmentation Using Temporal Gradient Pseudo-Labelscited by 7
  25. 2023Unsupervised Speech Recognition with N-Skipgram and Positional Unigram Matchingcited by 5
  26. 2023A Theory of Unsupervised Speech Recognitioncited by 4
  27. 2022Bridging Speech and Textual Pre-trained Models with Unsupervised ASRcited by 3
  28. 2024REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASRcited by 3
  29. 2024Towards Unsupervised Speech Recognition Without Pronunciation Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026findsylls: A Language-Agnostic Toolkit for Syllable-Level Speech Tokenization and Embeddingcited by 0
  2. 2026Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenizationcited by 0