mlsanityData to 14 Oct 2026

Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition

arXiv 2201.11207 ↗First posted 26 Jan 2022cs.SD cs.CL eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

The high cost of data acquisition makes Automatic Speech Recognition (ASR) model training problematic for most existing languages, including languages that do not even have a written script, or for which the phone inventories remain unknown. Past works explored multilingual training, transfer learning, as well as zero-shot learning in order to build ASR systems for these low-resource languages. While it has been shown that the pooling of resources from multiple languages is helpful, we have not yet seen a successful application of an ASR model to a language unseen during training. A crucial step in the adaptation of ASR from seen to unseen languages is the creation of the phone inventory of the unseen language. The ultimate goal of our work is to build the phone inventory of a language unseen during training in an unsupervised way without any knowledge about the language. In this paper, we 1) investigate the influence of different factors (i.e., model architecture, phonotactic model, type of speech representation) on phone recognition in an unknown language; 2) provide an analysis of which phones transfer well across languages and which do not in order to understand the limitations of and areas for further improvement for automatic phone inventory creation; and 3) present different methods to build a phone inventory of an unseen language in an unsupervised way. To that end, we conducted mono-, multi-, and crosslingual experiments on a set of 13 phonetically diverse languages and several in-depth analyses. We found a number of universal phone tokens (IPA symbols) that are well-recognized cross-linguistically. Through a detailed analysis of results, we conclude that unique sounds, similar sounds, and tone languages remain a major challenge for phonetic inventory discovery.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 27 of its 110 references that are in the map. First posted on arXiv 26 Jan 2022.

Builds on

27 of 110 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  4. 2015Attention-Based Models for Speech Recognitioncited by 401
  5. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 257
  6. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 235
  7. 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 211
  8. 2021Unsupervised Speech Recognitioncited by 148
  9. 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 93
  10. 2017Multilingual Speech Recognition With A Single End-To-End Modelcited by 87
  11. 2020Universal Phone Recognition with a Multilingual Allophone Systemcited by 81
  12. 2017The Zero Resource Speech Challenge 2017cited by 45
15 more
  1. 2019VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019cited by 25
  2. 2018Sequence-based Multi-lingual Low Resource Speech Recognitioncited by 20
  3. 2021Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptationcited by 19
  4. 2020Multilingual Jointly Trained Acoustic and Written Word Embeddingscited by 16
  5. 2018Transfer learning of language-independent end-to-end ASR with language model fusioncited by 10
  6. 2020That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languagescited by 10
  7. 2020How Phonotactics Affect Multilingual and Zero-shot ASR Performancecited by 10
  8. 2019Bayesian Subspace Hidden Markov Model for Acoustic Unit Discoverycited by 9
  9. 2020A Hierarchical Subspace Model for Language-Attuned Acoustic Unit Discoverycited by 9
  10. 2016Unsupervised word segmentation and lexicon discovery using acoustic word embeddingscited by 8
  11. 2020Acoustic span embeddings for multilingual query-by-example searchcited by 7
  12. 2017Multilingual Training and Cross-lingual Adaptation on CTC-based Acoustic Modelcited by 6
  13. 2018Multilingual and Unsupervised Subword Modeling for Zero-Resource Languagescited by 6
  14. 2018Multilingual bottleneck features for subword modeling in zero-resource languagescited by 3
  15. 2021Unsupervised Acoustic Unit Discovery by Leveraging a Language-Independent Subword Discriminative Feature Representationcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 0; 2025: 1.

5 papers.

  1. 2023Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalizationcited by 28
  2. 2023Language-Universal Phonetic Representation in Multilingual Speech Pretraining for Low-Resource Speech Recognitioncited by 3
  3. 2023Language-universal phonetic encoder for low-resource speech recognitioncited by 2
  4. 2023Improving grapheme-to-phoneme conversion by learning pronunciations from speech recordingscited by 2
  5. 2025MauBERT: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discoverycited by 1