# STTATTS: Unified Speech-To-Text And Text-To-Speech Model

arXiv: https://arxiv.org/abs/2410.18607 · HTML: https://mlsanity.com/paper/2410.18607/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2024.

## Built on by (most cited first, 3 of 3)

- [ArVoice: A Multi-Speaker Dataset for Arabic Speech Synthesis](https://mlsanity.com/paper/2505.20506/) (2025)
- [UniVoice: Unifying Autoregressive ASR and Flow-Matching based TTS with Large Language Models](https://mlsanity.com/paper/2510.04593/) (2025)
- [Benchmarking NLP-supported Language Sample Analysis for Swiss Children's Speech](https://mlsanity.com/paper/2504.00780/) (2025)

## Abstract (arXiv)

Speech recognition and speech synthesis models are typically trained separately, each with its own set of learning objectives, training data, and model parameters, resulting in two distinct large networks. We propose a parameter-efficient approach to learning ASR and TTS jointly via a multi-task learning objective and shared parameters. Our evaluation demonstrates that the performance of our multi-task model is comparable to that of individually trained models while significantly saving computational and memory costs (~50% reduction in the total number of parameters required for the two tasks combined). We experiment with English as a resource-rich language, and Arabic as a relatively low-resource language due to shortage of TTS data. Our models are trained with publicly available data, and both the training code and model checkpoints are openly available for further research.