mlsanityData to 14 Oct 2026

An Empirical Recipe for Universal Phone Recognition

arXiv 2603.29042 ↗First posted 30 Mar 2026cs.CL cs.LG cs.SDcited by 5

Not on the map. Why?

Abstract (arXiv)

Phone recognition (PR) is a key enabler of multilingual and low-resource speech processing tasks, yet robust performance remains elusive. Highly performant English-focused models do not generalize across languages, while multilingual models underutilize pretrained representations. It also remains unclear how data scale, architecture, and training objective contribute to multilingual PR. We present PhoneticXEUS -- trained on large-scale multilingual data and achieving state-of-the-art performance on both multilingual (17.7% PFER) and accented English speech (10.6% PFER). Through controlled ablations with evaluations across 100+ languages under a unified scheme, we empirically establish our training recipe and quantify the impact of SSL representations, data scale, and loss objectives. In addition, we analyze error patterns across language families, accented speech, and articulatory features. All data and code are released openly at https://github.com/changelinglab/PhoneticXeus

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 35 of its 49 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

35 of 49 references.

References · 35

  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2025Qwen3-Omni Technical Reportcited by 497
  4. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  5. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  6. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 257
  7. 2021Intermediate Loss Regularization for CTC-based Speech Recognitioncited by 96
  8. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 92
  9. 2020Universal Phone Recognition with a Multilingual Allophone Systemcited by 81
  10. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 80
  11. 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
  12. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 50
23 more
  1. 2021The Zero Resource Speech Challenge 2021: Spoken language modellingcited by 47
  2. 2019The Zero Resource Speech Challenge 2019: TTS without Tcited by 38
  3. 2022Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challengecited by 34
  4. 2023Improving Massively Multilingual ASR With Auxiliary CTC Objectivescited by 31
  5. 2019Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networkscited by 24
  6. 2018Hierarchical Multi Task Learning With CTCcited by 20
  7. 2025ZIPA: A family of efficient models for multilingual phone recognitioncited by 19
  8. 2024CR-CTC: Consistency regularization on CTC for improved speech recognitioncited by 17
  9. 2023Universal Automatic Phonetic Transcription into the International Phonetic Alphabetcited by 16
  10. 2020Towards Zero-shot Learning for Automatic Phonemic Transcriptioncited by 14
  11. 2023The taste of IPA: Towards open-vocabulary keyword spotting and forced alignment in any languagecited by 14
  12. 2025POWSM: A Phonetic Open Whisper-Style Speech Foundation Modelcited by 14
  13. 2023Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributescited by 13
  14. 2026[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmeticcited by 12
  15. 2021Multilingual and crosslingual speech recognition using phonological-vector based phone embeddingscited by 11
  16. 2020That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languagescited by 10
  17. 2021Differentiable Allophone Graphs for Language-Universal Speech Recognitioncited by 10
  18. 2024Phonetic Segmentation of the UCLA Phonetics Lab Archivecited by 9
  19. 2026Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspacescited by 8
  20. 2021Tusom2021: A Phonetically Transcribed Speech Dataset from an Endangered Language for Universal Phone Recognition Experimentscited by 5
  21. 2026PRiSM: Benchmarking Phone Realization in Speech Modelscited by 4
  22. 2026Multilingual Dysarthric Speech Assessment Using Universal Phone Recognition and Language-Specific Phonemic Contrast Modelingcited by 4
  23. 2026HuPER: A Human-Inspired Framework for Phonetic Perceptioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026BranchShine: Compact Raw-Audio-to-IPA Transcription with a RoPE E-Branchformer Encodercited by 1
  2. 2026Phone Segmentation and Recognition through Phonological Activation Mappingcited by 0
  3. 2026BranchShine-CR: Compact Multilingual IPA Transcription with Self-Conditioned CTC and Consistency Regularizationcited by 0
  4. 2026Training-Free Pronunciation Transcription via Text-Constrained Acoustic Rescoringcited by 0
  5. 2026Phonological Interference in Multilingual Speech Modelscited by 0