# AIPNet: Generative Adversarial Pre-training of Accent-invariant Networks for End-to-end Speech Recognition

arXiv: https://arxiv.org/abs/1911.11935 · HTML: https://mlsanity.com/paper/1911.11935/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2019.

## Built on by (most cited first, 11 of 11)

- [AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASR](https://mlsanity.com/paper/2310.00274/) (2023)
- [SpeechNet: A Universal Modularized Model for Speech Processing Tasks](https://mlsanity.com/paper/2105.03070/) (2021)
- [Accented Speech Recognition: A Survey](https://mlsanity.com/paper/2104.10747/) (2021)
- [Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing](https://mlsanity.com/paper/2211.01522/) (2022)
- [AfriNames: Most ASR models "butcher" African Names](https://mlsanity.com/paper/2306.00253/) (2023)
- [Improving Accented Speech Recognition using Data Augmentation based on Unsupervised Text-to-Speech Synthesis](https://mlsanity.com/paper/2407.04047/) (2024)
- [Fine-tuning of Pre-trained End-to-end Speech Recognition with Generative Adversarial Networks](https://mlsanity.com/paper/2103.13329/) (2021)
- [INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech Recognition](https://mlsanity.com/paper/2305.16371/) (2023)
- [Improving Cross-Lingual Reading Comprehension with Self-Training](https://mlsanity.com/paper/2105.03627/) (2021)
- [MADI: Inter-domain Matching and Intra-domain Discrimination for Cross-domain Speech Recognition](https://mlsanity.com/paper/2302.11224/) (2023)
- [Synthetic Cross-accent Data Augmentation for Automatic Speech Recognition](https://mlsanity.com/paper/2303.00802/) (2023)

## Abstract (arXiv)

As one of the major sources in speech variability, accents have posed a grand challenge to the robustness of speech recognition systems. In this paper, our goal is to build a unified end-to-end speech recognition system that generalizes well across accents. For this purpose, we propose a novel pre-training framework AIPNet based on generative adversarial nets (GAN) for accent-invariant representation learning: Accent Invariant Pre-training Networks. We pre-train AIPNet to disentangle accent-invariant and accent-specific characteristics from acoustic features through adversarial training on accented data for which transcriptions are not necessarily available. We further fine-tune AIPNet by connecting the accent-invariant module with an attention-based encoder-decoder model for multi-accent speech recognition. In the experiments, our approach is compared against four baselines including both accent-dependent and accent-independent models. Experimental results on 9 English accents show that the proposed approach outperforms all the baselines by 2.3 ~ 4.5% relative reduction on average WER when transcriptions are available in all accents and by 1.6 ~ 6.1% relative reduction when transcriptions are only available in US accent.