# Complete reconstruction of the tongue contour through acoustic to articulatory inversion using real-time MRI data

arXiv: https://arxiv.org/abs/2411.02037 · HTML: https://mlsanity.com/paper/2411.02037/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models.
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 4 Nov 2024.

## Placed in

- Speech and audio language models / [Vocoders and speech synthesis](https://mlsanity.com/speech/vocoders-and-speech-synthesis/)

## Builds on (most cited first, 5 of 5)

- [Speaker-Independent Acoustic-to-Articulatory Speech Inversion](https://mlsanity.com/paper/2302.06774/) (2023)
- [The Secret Source : Incorporating Source Features to Improve Acoustic-to-Articulatory Speech Inversion](https://mlsanity.com/paper/2210.16450/) (2022)
- [Acoustic-to-articulatory Speech Inversion with Multi-task Learning](https://mlsanity.com/paper/2205.13755/) (2022)
- [Speaker dependent acoustic-to-articulatory inversion using real-time MRI of the vocal tract](https://mlsanity.com/paper/2008.02098/) (2020)
- [Two-Stream Joint-Training for Speaker Independent Acoustic-to-Articulatory Inversion](https://mlsanity.com/paper/2302.13273/) (2023)

## Abstract (arXiv)

Acoustic articulatory inversion is a major processing challenge, with a wide range of applications from speech synthesis to feedback systems for language learning and rehabilitation. In recent years, deep learning methods have been applied to the inversion of less than a dozen geometrical positions corresponding to sensors glued to easily accessible articulators. It is therefore impossible to know the shape of the whole tongue from root to tip. In this work, we use high-quality real-time MRI data to track the contour of the tongue. The data used to drive the inversion are therefore the unstructured speech signal and the tongue contours. Several architectures relying on a Bi-MSTM including or not an autoencoder to reduce the dimensionality of the latent space, using or not the phonetic segmentation have been explored. The results show that the tongue contour can be recovered with a median accuracy of 2.21 mm (or 1.37 pixel) taking a context of 1 MFCC frame (static, delta and double-delta cepstral features).