# Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition arXiv: https://arxiv.org/abs/2201.11207 ยท HTML: https://mlsanity.com/paper/2201.11207/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Builds on 27 of its 110 references that are in the map. - First posted on arXiv 26 Jan 2022. ## Builds on (most cited first, 27 of 27) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018) - [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015) - [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016) - [Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss](https://mlsanity.com/paper/2002.02562/) (2020) - [A Comparative Study on Transformer vs RNN in Speech Applications](https://mlsanity.com/paper/1909.06317/) (2019) - [Unsupervised Speech Recognition](https://mlsanity.com/paper/2105.11084/) (2021) - [Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model](https://mlsanity.com/paper/1909.05330/) (2019) - [Multilingual Speech Recognition With A Single End-To-End Model](https://mlsanity.com/paper/1711.01694/) (2017) - [Universal Phone Recognition with a Multilingual Allophone System](https://mlsanity.com/paper/2002.11800/) (2020) - [The Zero Resource Speech Challenge 2017](https://mlsanity.com/paper/1712.04313/) (2017) - [VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019](https://mlsanity.com/paper/1905.11449/) (2019) - [Sequence-based Multi-lingual Low Resource Speech Recognition](https://mlsanity.com/paper/1802.07420/) (2018) - [Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptation](https://mlsanity.com/paper/2103.10731/) (2021) - [Multilingual Jointly Trained Acoustic and Written Word Embeddings](https://mlsanity.com/paper/2006.14007/) (2020) - [Transfer learning of language-independent end-to-end ASR with language model fusion](https://mlsanity.com/paper/1811.02134/) (2018) - [That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languages](https://mlsanity.com/paper/2005.08118/) (2020) - [How Phonotactics Affect Multilingual and Zero-shot ASR Performance](https://mlsanity.com/paper/2010.12104/) (2020) - [Bayesian Subspace Hidden Markov Model for Acoustic Unit Discovery](https://mlsanity.com/paper/1904.03876/) (2019) - [A Hierarchical Subspace Model for Language-Attuned Acoustic Unit Discovery](https://mlsanity.com/paper/2011.03115/) (2020) - [Unsupervised word segmentation and lexicon discovery using acoustic word embeddings](https://mlsanity.com/paper/1603.02845/) (2016) - [Acoustic span embeddings for multilingual query-by-example search](https://mlsanity.com/paper/2011.11807/) (2020) - [Multilingual Training and Cross-lingual Adaptation on CTC-based Acoustic Model](https://mlsanity.com/paper/1711.10025/) (2017) - [Multilingual and Unsupervised Subword Modeling for Zero-Resource Languages](https://mlsanity.com/paper/1811.04791/) (2018) - [Multilingual bottleneck features for subword modeling in zero-resource languages](https://mlsanity.com/paper/1803.08863/) (2018) - [Unsupervised Acoustic Unit Discovery by Leveraging a Language-Independent Subword Discriminative Feature Representation](https://mlsanity.com/paper/2104.00994/) (2021) ## Built on by (most cited first, 5 of 5) - [Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization](https://mlsanity.com/paper/2304.14535/) (2023) - [Language-Universal Phonetic Representation in Multilingual Speech Pretraining for Low-Resource Speech Recognition](https://mlsanity.com/paper/2305.11569/) (2023) - [Language-universal phonetic encoder for low-resource speech recognition](https://mlsanity.com/paper/2305.11576/) (2023) - [Improving grapheme-to-phoneme conversion by learning pronunciations from speech recordings](https://mlsanity.com/paper/2307.16643/) (2023) - [MauBERT: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery](https://mlsanity.com/paper/2512.19612/) (2025) ## Abstract (arXiv) The high cost of data acquisition makes Automatic Speech Recognition (ASR) model training problematic for most existing languages, including languages that do not even have a written script, or for which the phone inventories remain unknown. Past works explored multilingual training, transfer learning, as well as zero-shot learning in order to build ASR systems for these low-resource languages. While it has been shown that the pooling of resources from multiple languages is helpful, we have not yet seen a successful application of an ASR model to a language unseen during training. A crucial step in the adaptation of ASR from seen to unseen languages is the creation of the phone inventory of the unseen language. The ultimate goal of our work is to build the phone inventory of a language unseen during training in an unsupervised way without any knowledge about the language. In this paper, we 1) investigate the influence of different factors (i.e., model architecture, phonotactic model, type of speech representation) on phone recognition in an unknown language; 2) provide an analysis of which phones transfer well across languages and which do not in order to understand the limitations of and areas for further improvement for automatic phone inventory creation; and 3) present different methods to build a phone inventory of an unseen language in an unsupervised way. To that end, we conducted mono-, multi-, and crosslingual experiments on a set of 13 phonetically diverse languages and several in-depth analyses. We found a number of universal phone tokens (IPA symbols) that are well-recognized cross-linguistically. Through a detailed analysis of results, we conclude that unique sounds, similar sounds, and tone languages remain a major challenge for phonetic inventory discovery.