# Synthesis of Tongue Motion and Acoustics from Text using a Multimodal Articulatory Database

arXiv: https://arxiv.org/abs/1612.09352 · HTML: https://mlsanity.com/paper/1612.09352/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Dec 2016.

## Built on by (most cited first, 1 of 1)

- [EMA2S: An End-to-End Multimodal Articulatory-to-Speech System](https://mlsanity.com/paper/2102.03786/) (2021)

## Abstract (arXiv)

We present an end-to-end text-to-speech (TTS) synthesis system that generates audio and synchronized tongue motion directly from text. This is achieved by adapting a 3D model of the tongue surface to an articulatory dataset and training a statistical parametric speech synthesis system directly on the tongue model parameters. We evaluate the model at every step by comparing the spatial coordinates of predicted articulatory movements against the reference data. The results indicate a global mean Euclidean distance of less than 2.8 mm, and our approach can be adapted to add an articulatory modality to conventional TTS applications without the need for extra data.