mlsanityData to 14 Oct 2026

Cross-Speaker Emotion Transfer for Low-Resource Text-to-Speech Using Non-Parallel Voice Conversion with Pitch-Shift Data Augmentation

arXiv 2204.10020 ↗First posted 21 Apr 2022eess.AS cs.LG cs.SDcited by 8

Not on the map. Why?

Abstract (arXiv)

Data augmentation via voice conversion (VC) has been successfully applied to low-resource expressive text-to-speech (TTS) when only neutral data for the target speaker are available. Although the quality of VC is crucial for this approach, it is challenging to learn a stable VC model because the amount of data is limited in low-resource scenarios, and highly expressive speech has large acoustic variety. To address this issue, we propose a novel data augmentation method that combines pitch-shifting and VC techniques. Because pitch-shift data augmentation enables the coverage of a variety of pitch dynamics, it greatly stabilizes training for both VC and TTS models, even when only 1,000 utterances of the target speaker's neutral data are available. Subjective test results showed that a FastSpeech 2-based emotional TTS system with the proposed method improved naturalness and emotional similarity compared with conventional methods.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 24 of its 43 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

24 of 43 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  4. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  5. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 340
  6. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
  7. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  8. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 173
  9. 2020Recent Developments on ESPnet Toolkit Boosted by Conformercited by 132
  10. 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
  11. 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
  12. 2020Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversioncited by 75
12 more
  1. 2018Learning latent representations for style control and transfer in end-to-end speech synthesiscited by 61
  2. 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
  3. 2019CycleGAN-VC2: Improved CycleGAN-based Non-parallel Voice Conversioncited by 46
  4. 2019Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokenscited by 44
  5. 2018Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesiscited by 30
  6. 2020Fine-grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesiscited by 29
  7. 2020Low-resource expressive text-to-speech using data augmentationcited by 26
  8. 2019Exploring Transfer Learning for Low Resource Emotional TTScited by 21
  9. 2020TTS-by-TTS: TTS-driven Data Augmentation for Fast and High-Quality Speech Synthesiscited by 15
  10. 2018Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent languagecited by 14
  11. 2021Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speechcited by 14
  12. 2022Cross-speaker style transfer for text-to-speech using data augmentationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 1; 2026: 1.

8 papers.

  1. 2023ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Modelscited by 14
  2. 2023Nonparallel Emotional Voice Conversion For Unseen Speaker-Emotion Pairs Using Dual Domain Adversarial Network & Virtual Domain Pairingcited by 9
  3. 2024Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settingscited by 7
  4. 2023CALLS: Japanese Empathetic Dialogue Speech Corpus of Complaint Handling and Attentive Listening in Customer Centercited by 3
  5. 2025Voice Conversion for Lombard Speaking Style with Implicit and Explicit Acoustic Feature Conditioningcited by 2
  6. 2023Unsupervised Pre-Training For Data-Efficient Text-to-Speech On Low Resource Languagescited by 1
  7. 2024Discrete Unit based Masking for Improving Disentanglement in Voice Conversioncited by 1
  8. 2026Semantics-Aware Generative Latent Data Augmentation for Learning in Low-Resource Domainscited by 0