# Acoustic-to-articulatory Speech Inversion with Multi-task Learning

arXiv: https://arxiv.org/abs/2205.13755 · HTML: https://mlsanity.com/paper/2205.13755/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2022.

## Built on by (most cited first, 2 of 2)

- [Speaker- and Text-Independent Estimation of Articulatory Movements and Phoneme Alignments from Speech](https://mlsanity.com/paper/2407.03132/) (2024)
- [Articulation-Informed ASR: Integrating Articulatory Features into ASR via Auxiliary Speech Inversion and Cross-Attention Fusion](https://mlsanity.com/paper/2510.08585/) (2025)

## Abstract (arXiv)

Multi-task learning (MTL) frameworks have proven to be effective in diverse speech related tasks like automatic speech recognition (ASR) and speech emotion recognition. This paper proposes a MTL framework to perform acoustic-to-articulatory speech inversion by simultaneously learning an acoustic to phoneme mapping as a shared task. We use the Haskins Production Rate Comparison (HPRC) database which has both the electromagnetic articulography (EMA) data and the corresponding phonetic transcriptions. Performance of the system was measured by computing the correlation between estimated and actual tract variables (TVs) from the acoustic to articulatory speech inversion task. The proposed MTL based Bidirectional Gated Recurrent Neural Network (RNN) model learns to map the input acoustic features to nine TVs while outperforming the baseline model trained to perform only acoustic to articulatory inversion.