# Enhancing Acoustic-to-Articulatory Inversion with Multi-Target Pretraining for Low-Resource Settings

arXiv: https://arxiv.org/abs/2607.01594 · HTML: https://mlsanity.com/paper/2607.01594/

## Key facts

- Cited by 1 papers in the map (data through 19 Oct 2026).
- Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models.
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 2 Jul 2026.

## Placed in

- Speech and audio language models / [Vocoders and speech synthesis](https://mlsanity.com/speech/vocoders-and-speech-synthesis/)

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [Estimating articulatory movements in speech production with transformer networks](https://mlsanity.com/paper/2104.05017/) (2021)
- [Improved acoustic-to-articulatory inversion using representations from pretrained self-supervised learning models](https://mlsanity.com/paper/2210.16871/) (2022)
- [Audio Data Augmentation for Acoustic-to-articulatory Speech Inversion using Bidirectional Gated RNNs](https://mlsanity.com/paper/2205.13086/) (2022)
- [Acoustic-to-articulatory Inversion based on Speech Decomposition and Auxiliary Feature](https://mlsanity.com/paper/2204.00873/) (2022)
- [Acoustic-to-articulatory Speech Inversion with Multi-task Learning](https://mlsanity.com/paper/2205.13755/) (2022)
- [Two-Stream Joint-Training for Speaker Independent Acoustic-to-Articulatory Inversion](https://mlsanity.com/paper/2302.13273/) (2023)
- [Acoustic-to-articulatory inversion for dysarthric speech: Are pre-trained self-supervised representations favorable?](https://mlsanity.com/paper/2309.01108/) (2023)

## Built on by (most cited first, 1 of 1)

- [Articulatory Source-Filter TTS: Physically Grounded Control through Vocal Tract Kinematics](https://mlsanity.com/paper/2610.00735/) (2026)

## Abstract (arXiv)

Acoustic-to-Articulatory Inversion (AAI) estimates vocal tract articulator movements from speech, benefiting tasks like ASR, speech synthesis, and speaker verification. While deep learning-based methods (CNNs, RNNs, Transformers) have advanced AAI, recent studies show that Self-Supervised Learning (SSL) features further enhance performance, particularly in low-resource settings. However, SSL feature extractors introduce inference latency and computational overhead. To address this, we propose a novel pretraining method leveraging three target representations-Phoneme Labels, Articulatory Feature Labels, and Critical-articulator Labels-eliminating the need for an SSL extractor during inference. We evaluate our approach against both baseline and SSL-based models across various data conditions. Results demonstrate that our method consistently improves AAI performance, particularly in low-resource scenarios, while significantly reducing inference costs without sacrificing accuracy.