mlsanityData to 7 Oct 2026

Exploration of Language Dependency for Japanese Self-Supervised Speech Representation Models

arXiv 2305.05201 ↗First posted 9 May 2023cs.CL cs.SD eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) has been dramatically successful not only in monolingual but also in cross-lingual settings. However, since the two settings have been studied individually in general, there has been little research focusing on how effective a cross-lingual model is in comparison with a monolingual model. In this paper, we investigate this fundamental question empirically with Japanese automatic speech recognition (ASR) tasks. First, we begin by comparing the ASR performance of cross-lingual and monolingual models for two different language tasks while keeping the acoustic domain as identical as possible. Then, we examine how much unlabeled data collected in Japanese is needed to achieve performance comparable to a cross-lingual model pre-trained with tens of thousands of hours of English and/or multilingual data. Finally, we extensively investigate the effectiveness of SSL in Japanese and demonstrate state-of-the-art performance on multiple ASR tasks. Since there is no comprehensive SSL study for Japanese, we hope this study will guide Japanese SSL research.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 36 references that are in the map. First posted on arXiv 9 May 2023.

Builds on

29 of 36 references.

References · 29

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  7. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  8. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  9. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  10. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  11. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  12. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
17 more
  1. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  2. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  3. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  4. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 340
  5. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  6. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 185
  7. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 148
  8. 2020Unsupervised pretraining transfers well across languagescited by 100
  9. 2017JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesiscited by 57
  10. 2020Learning Robust and Multilingual Speech Representationscited by 39
  11. 2022Deep versus Wide: An Analysis of Student Architectures for Task-Agnostic Knowledge Distillation of Self-Supervised Speech Modelscited by 28
  12. 2022Improving non-autoregressive end-to-end speech recognition with pre-trained acoustic and language modelscited by 21
  13. 2021Don't speak too fast: The impact of data bias on self-supervised speech modelscited by 20
  14. 2022Investigating Self-supervised Pretraining Frameworks for Pathological Speech Recognitioncited by 15
  15. 2022Wav2Vec-Aug: Improved self-supervised training with limited datacited by 9
  16. 2021Construction of a Large-scale Japanese ASR Corpus on TV Recordingscited by 8
  17. 2021A Comparative Study on Neural Architectures and Training Methods for Japanese Speech Recognitioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 1; 2026: 1.

3 papers.

  1. 2023SpeechGLUE: How Well Can Self-Supervised Speech Models Capture Linguistic Knowledge?cited by 9
  2. 2025Self-supervised learning of speech representations with Dutch archival datacited by 0
  3. 2026Dissecting Sensitivity to Training Language in Self-Supervised Speech Learning Using Neural Audio Codec Tokenscited by 0