A Pilot Study of Applying Sequence-to-Sequence Voice Conversion to Evaluate the Intelligibility of L2 Speech Using a Native Speaker's Shadowings
Not on the map. Why?
Abstract (arXiv)
Utterances by L2 speakers can be unintelligible due to mispronunciation and improper prosody. In computer-aided language learning systems, textual feedback is often provided using a speech recognition engine. However, an ideal form of feedback for L2 speakers should be so fine-grained that it enables them to detect and diagnose unintelligible parts of L2 speakers' utterances. Inspired by language teachers who correct students' pronunciation through a voice-to-voice process, this pilot study utilizes a unique semi-parallel dataset composed of non-native speakers' (L2) reading aloud, shadowing of native speakers (L1) and their script-shadowing utterances. We explore the technical possibility of replicating the process of an L1 speaker's shadowing L2 speech using Voice Conversion techniques, to create a virtual shadower system. Experimental results demonstrate the feasibility of the VC system in simulating L1's shadowing behavior. The output of the virtual shadower system shows a reasonable similarity to the real L1 shadowing utterances in both linguistic and acoustic aspects.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 35 references that are in the map. First posted on arXiv 3 Oct 2024.
Builds on
20 of 35 references.
References · 20
- 2017Attention Is All You Needcited by 59,932
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
- 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 119
- 2023Can Generative Large Language Models Perform ASR Error Correction?cited by 56
- 2020Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modelingcited by 42
- 2020Joint Contextual Modeling for ASR Correction and Language Understandingcited by 30
- 2019Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretrainingcited by 29
8 more
- 2020FastSVC: Fast Cross-Domain Singing Voice Conversion with Feature-wise Linear Modulationcited by 21
- 2022Speaking Style Conversion in the Waveform Domain Using Discrete Self-Supervised Unitscited by 15
- 2020Pretraining Techniques for Sequence-to-Sequence Voice Conversioncited by 13
- 2022A Comparative Study of Self-supervised Speech Representation Based Voice Conversioncited by 9
- 2021Non-autoregressive sequence-to-sequence voice conversioncited by 7
- 2023Evaluating Methods for Ground-Truth-Free Foreign Accent Conversioncited by 4
- 2023Automatic Speech Recognition of Non-Native Child Speech for Language Learning Applicationscited by 3
- 2023AAS-VC: On the Generalization Ability of Automatic Alignment Search based Non-autoregressive Sequence-to-sequence Voice Conversioncited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.