mlsanityData to 7 Oct 2026

Simple and Effective Zero-shot Cross-lingual Phoneme Recognition

arXiv 2109.11680 ↗First posted 23 Sep 2021cs.CL cs.LG cs.SDcited by 79

Not on the map. Why?

Abstract (arXiv)

Recent progress in self-training, self-supervised pretraining and unsupervised learning enabled well performing speech recognition systems without any labeled data. However, in many cases there is labeled data available for related languages which is not utilized by these methods. This paper extends previous work on zero-shot cross-lingual transfer learning by fine-tuning a multilingually pretrained wav2vec 2.0 model to transcribe unseen languages. This is done by mapping phonemes of the training languages to the target language using articulatory features. Experiments show that this simple method significantly outperforms prior work which introduced task-specific architectures and used only part of a monolingually pretrained model.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 23 Sep 2021.

Builds on

30 of 39 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  9. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  10. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  11. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  12. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
18 more
  1. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 185
  2. 2021Unsupervised Speech Recognitioncited by 147
  3. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 139
  4. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  5. 2020Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameterscited by 110
  6. 2020Self-training and Pre-training are Complementary for Speech Recognitioncited by 98
  7. 2020Universal Phone Recognition with a Multilingual Allophone Systemcited by 81
  8. 2020Iterative Pseudo-Labeling for Speech Recognitioncited by 81
  9. 2018The challenge of realistic music generation: modelling raw audio at scalecited by 63
  10. 2020SlimIPL: Language-Model-Free Iterative Pseudo-Labelingcited by 54
  11. 2018Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speechcited by 52
  12. 2018wav2letter++: The Fastest Open-source Speech Recognition Systemcited by 42
  13. 2020Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognitioncited by 34
  14. 2018Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddingscited by 22
  15. 2018Sequence-based Multi-lingual Low Resource Speech Recognitioncited by 20
  16. 2020Towards Zero-shot Learning for Automatic Phonemic Transcriptioncited by 14
  17. 2021Differentiable Allophone Graphs for Language-Universal Speech Recognitioncited by 10
  18. 2021Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource languagecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 12; 2024: 12; 2025: 21; 2026: 28.

79 papers.

Sort:

By area:

  1. 2024GaussianTalker: Speaker-specific Talking Head Synthesis via 3D Gaussian Splattingcited by 24
  2. 2023Adapting Multilingual Speech Representation Model for a New, Underresourced Language through Multilingual Fine-tuning and Continued Pretrainingcited by 18
  3. 2025ZIPA: A family of efficient models for multilingual phone recognitioncited by 18
  4. 2024Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervisioncited by 17
  5. 2023Universal Automatic Phonetic Transcription into the International Phonetic Alphabetcited by 16
  6. 2023The taste of IPA: Towards open-vocabulary keyword spotting and forced alignment in any languagecited by 14
  7. 2025POWSM: A Phonetic Open Whisper-Style Speech Foundation Modelcited by 14
  8. 2023Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributescited by 13
  9. 2022Combining Spectral and Self-Supervised Features for Low Resource Speech Recognition and Translationcited by 11
  10. 2026[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmeticcited by 11
  11. 2024DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignmentcited by 8
  12. 2024Large Language Models for Dysfluency Detection in Stuttered Speechcited by 8
38 more
  1. 2022Out of Thin Air: Is Zero-Shot Cross-Lingual Keyword Detection Better Than Unsupervised?cited by 7
  2. 2022PANCETTA: Phoneme Aware Neural Completion to Elicit Tongue Twisters Automaticallycited by 6
  3. 2023READ Avatars: Realistic Emotion-controllable Audio Driven Avatarscited by 6
  4. 2023Multilingual Word Error Rate Estimation: e-WER3cited by 5
  5. 2023SelfVC: Voice Conversion With Iterative Refinement using Self Transformationscited by 5
  6. 2023Optimizing Two-Pass Cross-Lingual Transfer Learning: Phoneme Recognition and Phoneme to Grapheme Translationcited by 5
  7. 2025Improving Cross-Lingual Phonetic Representation of Low-Resource Languages Through Language Similarity Analysiscited by 5
  8. 2025Dysfluent WFST: A Framework for Zero-Shot Speech Dysfluency Transcription and Detectioncited by 5
  9. 2025PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysiscited by 5
  10. 2026An Empirical Recipe for Universal Phone Recognitioncited by 5
  11. 2022Applying wav2vec2 for Speech Recognition on Bengali Common Voices Datasetcited by 4
  12. 2023Uncertainty as a Predictor: Leveraging Self-Supervised Learning for Zero-Shot MOS Predictioncited by 4
  13. 2025Articulatory Feature Prediction from Surface EMG during Speech Productioncited by 4
  14. 2025Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysiscited by 4
  15. 2026PRiSM: Benchmarking Phone Realization in Speech Modelscited by 4
  16. 2026Quantifying Speaker Embedding Phonological Rule Interactions in Accented Speech Synthesiscited by 4
  17. 2026Multilingual Dysarthric Speech Assessment Using Universal Phone Recognition and Language-Specific Phonemic Contrast Modelingcited by 4
  18. 2024Automatic Speech Recognition (ASR) for the Diagnosis of pronunciation of Speech Sound Disorders in Korean childrencited by 3
  19. 2024LAMA-UT: Language Agnostic Multilingual ASR through Orthography Unification and Language-Specific Transliterationcited by 3
  20. 2025Towards disentangling the contributions of articulation and acoustics in multimodal phoneme recognitioncited by 3
  21. 2025LLM-based phoneme-to-grapheme for phoneme-based speech recognitioncited by 3
  22. 2026HuPER: A Human-Inspired Framework for Phonetic Perceptioncited by 3
  23. 2021Injecting Text and Cross-lingual Supervision in Few-shot Learning from Self-Supervised Modelscited by 2
  24. 2025Enhancing GOP in CTC-Based Mispronunciation Detection with Phonological Knowledgecited by 2
  25. 2025On the Relationship between Accent Strength and Articulatory Featurescited by 2
  26. 2025A Bridge from Audio to Video: Phoneme-Viseme Alignment Allows Every Face to Speak Multiple Languagescited by 2
  27. 2026Multilingual Phonological Feature Recognition with Self-Supervised Speech Modelscited by 2
  28. 2023DistilXLSR: A Light Weight Cross-Lingual Speech Representation Modelcited by 1
  29. 2024Reading Miscue Detection in Primary School through Automatic Speech Recognitioncited by 1
  30. 2024TalkLoRA: Low-Rank Adaptation for Speech-Driven Animationcited by 1
  31. 2024Efficiently Identifying Low-Quality Language Subsets in Multilingual Datasets: A Case Study on a Large-Scale Multilingual Audio Datasetcited by 1
  32. 2025LID Models are Actually Accent Classifiers: Implications and Solutions for LID on Accented Speechcited by 1
  33. 2025ViCocktail: Automated Multi-Modal Data Collection for Vietnamese Audio-Visual Speech Recognitioncited by 1
  34. 2025Evaluating Logit-Based GOP Scores for Mispronunciation Detectioncited by 1
  35. 2025Smooth Operators: LLMs Translating Imperfect Hints into Disfluency-Rich Transcriptscited by 1
  36. 2025Whisper based Cross-Lingual Phoneme Recognition between Vietnamese and Englishcited by 1
  37. 2025Building Tailored Speech Recognizers for Japanese Speaking Assessmentcited by 1
  38. 2025On the Cross-lingual Transferability of Pre-trained wav2vec2-based Modelscited by 1

50 most cited of 79 All 79 (JSON)