mlsanityData to 14 Oct 2026

ADAPTERMIX: Exploring the Efficacy of Mixture of Adapters for Low-Resource TTS Adaptation

arXiv 2305.18028 ↗First posted 29 May 2023cs.SD cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

There are significant challenges for speaker adaptation in text-to-speech for languages that are not widely spoken or for speakers with accents or dialects that are not well-represented in the training data. To address this issue, we propose the use of the "mixture of adapters" method. This approach involves adding multiple adapters within a backbone-model layer to learn the unique characteristics of different speakers. Our approach outperforms the baseline, with a noticeable improvement of 5% observed in speaker preference tests when using only one minute of data for each new speaker. Moreover, following the adapter paradigm, we fine-tune only the adapter parameters (11% of the total model parameters). This is a significant achievement in parameter-efficient speaker adaptation, and one of the first models of its kind. Overall, our proposed approach offers a promising solution to the speech synthesis techniques, particularly for adapting to speakers from diverse backgrounds.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 28 of its 36 references that are in the map. First posted on arXiv 29 May 2023.

Builds on

28 of 36 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Layer Normalizationcited by 5,669
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  7. 2022Mixture-of-Experts with Expert Choice Routingcited by 623
  8. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
  9. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  10. 2018Neural Voice Cloning with a Few Samplescited by 100
  11. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  12. 2023AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Modelscited by 84
16 more
  1. 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
  2. 2017Deep Speaker: an End-to-End Neural Speaker Embedding Systemcited by 58
  3. 2023A Review of Deep Learning Techniques for Speech Processingcited by 52
  4. 2018Sample Efficient Adaptive Text-to-Speechcited by 45
  5. 2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
  6. 2021Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speechcited by 22
  7. 2021Expressive Text-to-Speech using Style Tagcited by 20
  8. 2020Hierarchical Prosody Modeling for Non-Autoregressive Speech Synthesiscited by 16
  9. 2022Multilingual Machine Translation with Hyper-Adapterscited by 15
  10. 2021Adapting TTS models For New Speakers using Transfer Learningcited by 12
  11. 2019High quality, lightweight and adaptable TTS using LPCNetcited by 11
  12. 2022Residual Adapters for Few-Shot Text-to-Speech Speaker Adaptationcited by 10
  13. 2021Hierarchical prosody modeling and control in non-autoregressive parallel neural TTScited by 9
  14. 2021Continual Speaker Adaptation for Text-to-Speech Synthesiscited by 5
  15. 2022TDASS: Target Domain Adaptation Speech Synthesis Framework for Multi-speaker Low-Resource TTScited by 5
  16. 2023MixPHM: Redundancy-Aware Parameter-Efficient Tuning for Low-Resource Visual Question Answeringcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 2; 2026: 1.

7 papers.

  1. 2024Efficient Fine-tuning of Audio Spectrogram Transformers via Soft Mixture of Adapterscited by 5
  2. 2024Lightweight Zero-shot Text-to-Speech with Mixture of Adapterscited by 2
  3. 2025Improving Recursive Transformers with Mixture of LoRAscited by 2
  4. 2024HyperTTS: Parameter Efficient Adaptation in Text to Speech using Hypernetworkscited by 1
  5. 2024Leveraging Parameter-Efficient Transfer Learning for Multi-Lingual Text-to-Speech Adaptationcited by 1
  6. 2025Unseen Speaker and Language Adaptation for Lightweight Text-To-Speech with Adapterscited by 0
  7. 2026ZeSTA: Zero-Shot TTS Augmentation with Domain-Conditioned Training for Data-Efficient Personalized Speech Synthesiscited by 0