# Unsupervised Acoustic Unit Discovery by Leveraging a Language-Independent Subword Discriminative Feature Representation

arXiv: https://arxiv.org/abs/2104.00994 · HTML: https://mlsanity.com/paper/2104.00994/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 2 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations](https://mlsanity.com/paper/1910.05453/) (2019)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [Unsupervised pretraining transfers well across languages](https://mlsanity.com/paper/2002.02848/) (2020)
- [Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge](https://mlsanity.com/paper/2005.09409/) (2020)
- [The Zero Resource Speech Challenge 2017](https://mlsanity.com/paper/1712.04313/) (2017)
- [The Zero Resource Speech Challenge 2020: Discovering discrete subword and word units](https://mlsanity.com/paper/2010.05967/) (2020)
- [The Zero Resource Speech Challenge 2019: TTS without T](https://mlsanity.com/paper/1904.11469/) (2019)
- [An embedded segmental K-means model for unsupervised segmentation and clustering of speech](https://mlsanity.com/paper/1703.08135/) (2017)
- [That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languages](https://mlsanity.com/paper/2005.08118/) (2020)
- [How Phonotactics Affect Multilingual and Zero-shot ASR Performance](https://mlsanity.com/paper/2010.12104/) (2020)
- [A Very Low Resource Language Speech Corpus for Computational Language Documentation Experiments](https://mlsanity.com/paper/1710.03501/) (2017)
- [Bayesian Subspace Hidden Markov Model for Acoustic Unit Discovery](https://mlsanity.com/paper/1904.03876/) (2019)
- [A Hierarchical Subspace Model for Language-Attuned Acoustic Unit Discovery](https://mlsanity.com/paper/2011.03115/) (2020)
- [Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling](https://mlsanity.com/paper/1906.07234/) (2019)
- [Bayesian Models for Unit Discovery on a Very Low Resource Language](https://mlsanity.com/paper/1802.06053/) (2018)
- [Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling](https://mlsanity.com/paper/1908.03538/) (2019)
- [Unsupervised Subword Modeling Using Autoregressive Pretraining and Cross-Lingual Phone-Aware Modeling](https://mlsanity.com/paper/2007.13002/) (2020)
- [The effectiveness of unsupervised subword modeling with autoregressive and cross-lingual phone-aware networks](https://mlsanity.com/paper/2012.09544/) (2020)

## Built on by (most cited first, 2 of 2)

- [Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition](https://mlsanity.com/paper/2201.11207/) (2022)
- [REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR](https://mlsanity.com/paper/2402.03988/) (2024)

## Abstract (arXiv)

This paper tackles automatically discovering phone-like acoustic units (AUD) from unlabeled speech data. Past studies usually proposed single-step approaches. We propose a two-stage approach: the first stage learns a subword-discriminative feature representation and the second stage applies clustering to the learned representation and obtains phone-like clusters as the discovered acoustic units. In the first stage, a recently proposed method in the task of unsupervised subword modeling is improved by replacing a monolingual out-of-domain (OOD) ASR system with a multilingual one to create a subword-discriminative representation that is more language-independent. In the second stage, segment-level k-means is adopted, and two methods to represent the variable-length speech segments as fixed-dimension feature vectors are compared. Experiments on a very low-resource Mboshi language corpus show that our approach outperforms state-of-the-art AUD in both normalized mutual information (NMI) and F-score. The multilingual ASR improved upon the monolingual ASR in providing OOD phone labels and in estimating the phone boundaries. A comparison of our systems with and without knowing the ground-truth phone boundaries showed a 16% NMI performance gap, suggesting that the current approach can significantly benefit from improved phone boundary estimation.