# Towards noise-robust speech inversion through multi-task learning with speech enhancement

arXiv: https://arxiv.org/abs/2601.14516 · HTML: https://mlsanity.com/paper/2601.14516/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models.
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Placed in

- Speech and audio language models / [Speech enhancement and noise suppression](https://mlsanity.com/speech/speech-enhancement-and-noise-suppression/)

## Builds on (most cited first, 11 of 11)

- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Phase-aware Speech Enhancement with Deep Complex U-Net](https://mlsanity.com/paper/1903.03107/) (2019)
- [Interspeech 2021 Deep Noise Suppression Challenge](https://mlsanity.com/paper/2101.01902/) (2021)
- [Investigating self-supervised learning for speech enhancement and separation](https://mlsanity.com/paper/2203.07960/) (2022)
- [Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speech](https://mlsanity.com/paper/2210.11723/) (2022)
- [Improving Speech Inversion Through Self-Supervised Embeddings and Enhanced Tract Variables](https://mlsanity.com/paper/2309.09220/) (2023)
- [Audio Data Augmentation for Acoustic-to-articulatory Speech Inversion using Bidirectional Gated RNNs](https://mlsanity.com/paper/2205.13086/) (2022)
- [Exploiting Consistency-Preserving Loss and Perceptual Contrast Stretching to Boost SSL-based Speech Enhancement](https://mlsanity.com/paper/2408.04773/) (2024)
- [Enhancing Acoustic-to-Articulatory Speech Inversion by Incorporating Nasality](https://mlsanity.com/paper/2506.09231/) (2025)
- [Acoustic to Articulatory Speech Inversion for Children with Velopharyngeal Insufficiency](https://mlsanity.com/paper/2509.09489/) (2025)

## Abstract (arXiv)

Recent studies demonstrate the effectiveness of Self Supervised Learning (SSL) speech representations for Speech Inversion (SI). However, applying SI in real-world scenarios remains challenging due to the pervasive presence of background noise. We propose a unified framework that integrates Speech Enhancement (SE) and SI models through shared SSL-based speech representations. In this framework, the SSL model is trained not only to support the SE module in suppressing noise but also to produce representations that are more informative for the SI task, allowing both modules to benefit from joint training. At a Signal-to-Noise Ratio of -5 db, our method for the SI task achieves relative improvements over the baseline of 80.95% under babble noise and 38.98% under non-babble noise, as measured by the average Pearson product-moment correlation across all estimated parameters.