Disentangling segmental and prosodic factors to non-native speech comprehensibility
Not on the map. Why?
Abstract (arXiv)
Current accent conversion (AC) systems do not disentangle the two main sources of non-native accent: segmental and prosodic characteristics. Being able to manipulate a non-native speaker's segmental and/or prosodic channels independently is critical to quantify how these two channels contribute to speech comprehensibility and social attitudes. We present an AC system that not only decouples voice quality from accent, but also disentangles the latter into its segmental and prosodic characteristics. The system is able to generate accent conversions that combine (1) the segmental characteristics from a source utterance, (2) the voice characteristics from a target utterance, and (3) the prosody of a reference utterance. We show that vector quantization of acoustic embeddings and removal of consecutive duplicated codewords allows the system to transfer prosody and improve voice similarity. We conduct perceptual listening tests to quantify the individual contributions of segmental features and prosody on the perceived comprehensibility of non-native speech. Our results indicate that, contrary to prior research in non-native speech, segmental features have a larger impact on comprehensibility than prosody. The proposed AC system may also be used to study how segmental and prosody cues affect social attitudes towards non-native speech.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 35 of its 63 references that are in the map. First posted on arXiv 20 Aug 2024.
Builds on
35 of 63 references.
References · 35
- 2017Attention Is All You Needcited by 59,932
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
- 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
- 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
- 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
- 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
23 more
- 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
- 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 189
- 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
- 2020An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learningcited by 111
- 2018Neural Voice Cloning with a Few Samplescited by 100
- 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
- 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
- 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
- 2020Unsupervised Speech Decomposition via Triple Information Bottleneckcited by 87
- 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 63
- 2018Learning latent representations for style control and transfer in end-to-end speech synthesiscited by 61
- 2021A Comparison of Discrete and Soft Speech Units for Improved Voice Conversioncited by 61
- 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 57
- 2020Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesiscited by 54
- 2019Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokenscited by 44
- 2020AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalizationcited by 44
- 2020VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net architecturecited by 34
- 2019Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretrainingcited by 29
- 2020Accent and Speaker Disentanglement in Many-to-many Voice Conversioncited by 15
- 2024What Do Self-Supervised Speech and Speaker Models Learn? New Findings From a Cross Model Layer-Wise Analysiscited by 10
- 2021Learning De-identified Representations of Prosody from Raw Audiocited by 8
- 2021An Improved StarGAN for Emotional Voice Conversion: Enhancing Voice Quality and Data Augmentationcited by 7
- 2021Learning Robust Latent Representations for Controllable Speech Synthesiscited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.