# SEAM: Spectral Embedding Alignment and Merging for Low-Resource TTS

arXiv: https://arxiv.org/abs/2610.26252 · HTML: https://mlsanity.com/paper/2610.26252/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models.
- Builds on 7 of its 12 references that are in the map.
- First posted on arXiv 19 Oct 2026.

## Placed in

- Speech and audio language models / [Vocoders and speech synthesis](https://mlsanity.com/speech/vocoders-and-speech-synthesis/)

## Builds on (most cited first, 7 of 7)

- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens](https://mlsanity.com/paper/2407.05407/) (2024)
- [YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone](https://mlsanity.com/paper/2112.02418/) (2021)
- [VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild](https://mlsanity.com/paper/2403.16973/) (2024)
- [Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech](https://mlsanity.com/paper/2111.04040/) (2021)
- [Facilitating large language model Russian adaptation with Learned Embedding Propagation](https://mlsanity.com/paper/2412.21140/) (2024)
- [Few-Shot Cross-Lingual TTS Using Transferable Phoneme Embedding](https://mlsanity.com/paper/2206.15427/) (2022)

## Abstract (arXiv)

Expanding the vocabulary of conditional text-to-speech (TTS) models typically requires extensive fine-tuning with paired audio, creating a major bottleneck for low-resource and cross-lingual adaptation. To address this, we introduce SEAM, a zero-audio initialization method that constructs embeddings for unseen tokens from known subtokens. Unlike standard pooling strategies that ignore sequential structure, SEAM applies a spectral filter to preserve temporal phonetic order and uses closed-form ridge regression to align the new embeddings with the pretrained conditioning manifold. Across three languages and two TTS architectures, SEAM approaches or matches baseline performance using up to four times less training data, reducing sample complexity by 75% compared to random initialization and mean pooling. Our approach provides a lightweight, reproducible, and theoretically grounded solution for vocabulary expansion in low-resource speech synthesis.