Language-universal phonetic encoder for low-resource speech recognition
Not on the map. Why?
Abstract (arXiv)
Multilingual training is effective in improving low-resource ASR, which may partially be explained by phonetic representation sharing between languages. In end-to-end (E2E) ASR systems, graphemes are often used as basic modeling units, however graphemes may not be ideal for multilingual phonetic sharing. In this paper, we leverage International Phonetic Alphabet (IPA) based language-universal phonetic model to improve low-resource ASR performances, for the first time within the attention encoder-decoder architecture. We propose an adaptation method on the phonetic IPA model to further improve the proposed approach on extreme low-resource languages. Experiments carried out on the open-source MLS corpus and our internal databases show our approach outperforms baseline monolingual models and most state-of-the-art works. Our main approach and adaptation are effective on extremely low-resource languages, even within domain- and language-mismatched scenarios.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 29 of its 37 references that are in the map. First posted on arXiv 19 May 2023.
Builds on
29 of 37 references.
References · 29
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
- 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
- 2015Listen, Attend and Spellcited by 543
- 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
- 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
- 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
- 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 211
- 2020Recent Developments on ESPnet Toolkit Boosted by Conformercited by 132
17 more
- 2020Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameterscited by 110
- 2017Multilingual Speech Recognition With A Single End-To-End Modelcited by 87
- 2021Scaling End-to-End Models for Large-Scale Multilingual ASRcited by 61
- 2018Bytes are All You Need: End-to-End Multilingual Speech Recognition and Synthesis with Bytescited by 54
- 2018wav2letter++: The Fastest Open-source Speech Recognition Systemcited by 42
- 2021MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognitioncited by 39
- 2021Joint Unsupervised and Supervised Training for Multilingual ASRcited by 39
- 2021Exploiting Adapters for Cross-lingual Low-resource Speech Recognitioncited by 35
- 2015Multilingual Language Processing From Bytescited by 26
- 2018Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languagescited by 26
- 2017Towards Language-Universal End-to-End Speech Recognitioncited by 14
- 2021XLST: Cross-lingual Self-training to Learn Multilingual Representation for Low Resource Speech Recognitioncited by 11
- 2020That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languagescited by 10
- 2020How Phonotactics Affect Multilingual and Zero-shot ASR Performancecited by 10
- 2022Hierarchical Softmax for End-to-End Low-resource Multilingual Speech Recognitioncited by 7
- 2017Multilingual Training and Cross-lingual Adaptation on CTC-based Acoustic Modelcited by 6
- 2022Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognitioncited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.