mlsanityData to 14 Oct 2026

Evaluating Self-supervised Speech Models on a Taiwanese Hokkien Corpus

arXiv 2312.06668 ↗First posted 6 Dec 2023cs.CL cs.SD eess.AScited by 7

Not on the map. Why?

Abstract (arXiv)

Taiwanese Hokkien is declining in use and status due to a language shift towards Mandarin in Taiwan. This is partly why it is a low resource language in NLP and speech research today. To ensure that the state of the art in speech processing does not leave Taiwanese Hokkien behind, we contribute a 1.5-hour dataset of Taiwanese Hokkien to ML-SUPERB's hidden set. Evaluating ML-SUPERB's suite of self-supervised learning (SSL) speech representations on our dataset, we find that model size does not consistently determine performance. In fact, certain smaller models outperform larger ones. Furthermore, linguistic alignment between pretraining data and the target language plays a crucial role.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

17 of 31 references.

References · 17

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  6. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  7. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  8. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  9. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  10. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  11. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  12. 2021Direct speech-to-speech translation with discrete unitscited by 149
5 more
  1. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  2. 2021Hybrid Spectrogram and Waveform Source Separationcited by 91
  3. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 71
  4. 2022Speech-to-Speech Translation For A Real-world Unwritten Languagecited by 25
  5. 2023Exploring Methods for Building Dialects-Mandarin Code-Mixing Corpora: A Case Study in Taiwanese Hokkiencited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 5.

7 papers.

  1. 2024ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasetscited by 27
  2. 2025CLiFT-ASR: A Cross-Lingual Fine-Tuning Framework for Low-Resource Taiwanese Hokkien Speech Recognitioncited by 2
  3. 2026Towards Comprehensive Semantic Speech Embeddings for Chinese Dialectscited by 1
  4. 2026Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Taskscited by 1
  5. 2026TaigiSpeech: A Low-Resource Real-World Speech Intent Dataset and Preliminary Results with Scalable Data Mining In-the-Wildcited by 1
  6. 2026Dissecting Sensitivity to Training Language in Self-Supervised Speech Learning Using Neural Audio Codec Tokenscited by 0
  7. 2026T-SANDHI: Tone Sandhi-aware Adaptive Network with Decoupled Hybrid Injection for Low-resource Taiwanese Hokkien Speech Recognitioncited by 0