mlsanityData to 14 Oct 2026

Improving Pronunciation and Accent Conversion through Knowledge Distillation And Synthetic Ground-Truth from Native TTS

arXiv 2410.14997 ↗First posted 19 Oct 2024cs.SD cs.AI eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

Previous approaches on accent conversion (AC) mainly aimed at making non-native speech sound more native while maintaining the original content and speaker identity. However, non-native speakers sometimes have pronunciation issues, which can make it difficult for listeners to understand them. Hence, we developed a new AC approach that not only focuses on accent conversion but also improves pronunciation of non-native accented speaker. By providing the non-native audio and the corresponding transcript, we generate the ideal ground-truth audio with native-like pronunciation with original duration and prosody. This ground-truth data aids the model in learning a direct mapping between accented and native speech. We utilize the end-to-end VITS framework to achieve high-quality waveform reconstruction for the AC task. As a result, our system not only produces audio that closely resembles native accents and while retaining the original speaker's identity but also improve pronunciation, as demonstrated by evaluation results.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 17 of its 32 references that are in the map. First posted on arXiv 19 Oct 2024.

Builds on

17 of 32 references.

References · 17

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2015Variational Inference with Normalizing Flowscited by 2,170
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  6. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  7. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  9. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 80
  10. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 64
  11. 2020Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modelingcited by 42
  12. 2020Relative Positional Encoding for Speech Recognition and Direct Translationcited by 23
5 more
  1. 2022Voice-preserving Zero-shot Multiple Accent Conversioncited by 17
  2. 2022TTS-Guided Training for Accent Conversion Without Parallel Datacited by 9
  3. 2022Zero-Shot Accent Conversion using Pseudo Siamese Disentanglement Networkcited by 7
  4. 2024Transfer the linguistic representations from TTS to accent conversion with non-parallel datacited by 5
  5. 2023Evaluating Methods for Ground-Truth-Free Foreign Accent Conversioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversioncited by 2
  2. 2026CosyAccent: Duration-Controllable Accent Normalization Using Source-Synthesis Training Datacited by 2
  3. 2025Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvementcited by 1
  4. 2026Controllable Accent Normalization via Discrete Diffusioncited by 0
  5. 2026TokAN: Accent Normalization Using Self-Supervised Speech Tokenscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Disentangling segmental and prosodic factors to non-native speech comprehensibility
  2. 2025Accent Normalization Using Self-Supervised Discrete Tokens with Non-Parallel Data
  3. 2025CLARITY: Contextual Linguistic Adaptation and Accent Retrieval for Dual-Bias Mitigation in Text-to-Speech Generation
  4. 2024Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation
  5. 2024DART: Disentanglement of Accent and Speaker Representation in Multispeaker Text-to-Speech
  6. 2025Language translation, and change of accent for speech-to-speech task using diffusion model
  7. 2022Data-augmented cross-lingual synthesis in a teacher-student framework
  8. 2025DiaMoE-TTS: A Unified IPA-Based Dialect TTS Framework with Mixture-of-Experts and Parameter-Efficient Zero-Shot Adaptation