# Improving Pronunciation and Accent Conversion through Knowledge Distillation And Synthetic Ground-Truth from Native TTS

arXiv: https://arxiv.org/abs/2410.14997 · HTML: https://mlsanity.com/paper/2410.14997/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 19 Oct 2024.

## Builds on (most cited first, 17 of 17)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion](https://mlsanity.com/paper/2210.15418/) (2022)
- [VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversion](https://mlsanity.com/paper/2106.10132/) (2021)
- [Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling](https://mlsanity.com/paper/2009.02725/) (2020)
- [Relative Positional Encoding for Speech Recognition and Direct Translation](https://mlsanity.com/paper/2005.09940/) (2020)
- [Voice-preserving Zero-shot Multiple Accent Conversion](https://mlsanity.com/paper/2211.13282/) (2022)
- [TTS-Guided Training for Accent Conversion Without Parallel Data](https://mlsanity.com/paper/2212.10204/) (2022)
- [Zero-Shot Accent Conversion using Pseudo Siamese Disentanglement Network](https://mlsanity.com/paper/2212.05751/) (2022)
- [Transfer the linguistic representations from TTS to accent conversion with non-parallel data](https://mlsanity.com/paper/2401.03538/) (2024)
- [Evaluating Methods for Ground-Truth-Free Foreign Accent Conversion](https://mlsanity.com/paper/2309.02133/) (2023)

## Built on by (most cited first, 5 of 5)

- [Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion](https://mlsanity.com/paper/2506.04013/) (2025)
- [CosyAccent: Duration-Controllable Accent Normalization Using Source-Synthesis Training Data](https://mlsanity.com/paper/2602.19166/) (2026)
- [Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement](https://mlsanity.com/paper/2506.16580/) (2025)
- [Controllable Accent Normalization via Discrete Diffusion](https://mlsanity.com/paper/2603.14275/) (2026)
- [TokAN: Accent Normalization Using Self-Supervised Speech Tokens](https://mlsanity.com/paper/2607.03928/) (2026)

## Abstract (arXiv)

Previous approaches on accent conversion (AC) mainly aimed at making non-native speech sound more native while maintaining the original content and speaker identity. However, non-native speakers sometimes have pronunciation issues, which can make it difficult for listeners to understand them. Hence, we developed a new AC approach that not only focuses on accent conversion but also improves pronunciation of non-native accented speaker. By providing the non-native audio and the corresponding transcript, we generate the ideal ground-truth audio with native-like pronunciation with original duration and prosody. This ground-truth data aids the model in learning a direct mapping between accented and native speech. We utilize the end-to-end VITS framework to achieve high-quality waveform reconstruction for the AC task. As a result, our system not only produces audio that closely resembles native accents and while retaining the original speaker's identity but also improve pronunciation, as demonstrated by evaluation results.