# Completely Unsupervised Speech Recognition By A Generative Adversarial Network Harmonized With Iteratively Refined Hidden Markov Models

arXiv: https://arxiv.org/abs/1904.04100 · HTML: https://mlsanity.com/paper/1904.04100/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2019.

## Built on by (most cited first, 8 of 8)

- [Unsupervised Automatic Speech Recognition: A Review](https://mlsanity.com/paper/2106.04897/) (2021)
- [REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR](https://mlsanity.com/paper/2402.03988/) (2024)
- [Bridging Speech and Textual Pre-trained Models with Unsupervised ASR](https://mlsanity.com/paper/2211.03025/) (2022)
- [Word Order Does Not Matter For Speech Recognition](https://mlsanity.com/paper/2110.05994/) (2021)
- [Enhancing Unsupervised Speech Recognition with Diffusion GANs](https://mlsanity.com/paper/2303.13559/) (2023)
- [Towards Unsupervised Speech Recognition at the Syllable-Level](https://mlsanity.com/paper/2510.03639/) (2025)
- [Towards Unsupervised Speech Recognition Without Pronunciation Models](https://mlsanity.com/paper/2406.08380/) (2024)
- [Sequence-Level Unsupervised Training in Speech Recognition: A Theoretical Study](https://mlsanity.com/paper/2603.02285/) (2026)

## Abstract (arXiv)

Producing a large annotated speech corpus for training ASR systems remains difficult for more than 95% of languages all over the world which are low-resourced, but collecting a relatively big unlabeled data set for such languages is more achievable. This is why some initial effort have been reported on completely unsupervised speech recognition learned from unlabeled data only, although with relatively high error rates. In this paper, we develop a Generative Adversarial Network (GAN) to achieve this purpose, in which a Generator and a Discriminator learn from each other iteratively to improve the performance. We further use a set of Hidden Markov Models (HMMs) iteratively refined from the machine generated labels to work in harmony with the GAN. The initial experiments on TIMIT data set achieve an phone error rate of 33.1%, which is 8.5% lower than the previous state-of-the-art.