mlsanityData to 14 Oct 2026

Unseen Speaker and Language Adaptation for Lightweight Text-To-Speech with Adapters

arXiv 2508.18006 ↗First posted 25 Aug 2025eess.AS cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper we investigate cross-lingual Text-To-Speech (TTS) synthesis through the lens of adapters, in the context of lightweight TTS systems. In particular, we compare the tasks of unseen speaker and language adaptation with the goal of synthesising a target voice in a target language, in which the target voice has no recordings therein. Results from objective evaluations demonstrate the effectiveness of adapters in learning language-specific and speaker-specific information, allowing pre-trained models to learn unseen speaker identities or languages, while avoiding catastrophic forgetting of the original model's speaker or language information. Additionally, to measure how native the generated voices are in terms of accent, we propose and validate an objective metric inspired by mispronunciation detection techniques in second-language (L2) learners. The paper also provides insights into the impact of adapter placement, configuration and the number of speakers used.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 25 Aug 2025.

Builds on

33 of 43 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2017Squeeze-and-Excitation Networkscited by 4,789
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  8. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  9. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  10. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  11. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  12. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
21 more
  1. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 354
  2. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 340
  3. 2017Generalized End-to-End Loss for Speaker Verificationcited by 293
  4. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 167
  5. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 74
  6. 2019Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloningcited by 66
  7. 2020Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speechcited by 63
  8. 2023TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudiocited by 59
  9. 2021SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Modelcited by 56
  10. 2021LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Searchcited by 28
  11. 2020An End-to-End Mispronunciation Detection System for L2 English Speech Leveraging Novel Anti-Phone Modelingcited by 18
  12. 2019Unsupervised Polyglot Text To Speechcited by 10
  13. 2022Residual Adapters for Few-Shot Text-to-Speech Speaker Adaptationcited by 10
  14. 2022Adapter-Based Extension of Multi-Speaker Text-to-Speech Model for New Speakerscited by 9
  15. 2020An Effective End-to-End Modeling Approach for Mispronunciation Detectioncited by 8
  16. 2023ADAPTERMIX: Exploring the Efficacy of Mixture of Adapters for Low-Resource TTS Adaptationcited by 7
  17. 2022Training Text-To-Speech Systems From Synthetic Data: A Practical Approach For Accent Transfer Taskscited by 6
  18. 2023Evaluating Parameter-Efficient Transfer Learning Approaches on SURE Benchmark for Speech Understandingcited by 5
  19. 2022Mix and Match: An Empirical Study on Training Corpus Composition for Polyglot Text-To-Speech (TTS)cited by 4
  20. 2022Cross-lingual Text-To-Speech with Flow-based Voice Conversion for Improved Pronunciationcited by 3
  21. 2025Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applicationscited by 1

Built on by

Not cited yet.