# Test-time adaptation for speech enhancement with an autoregressive speech prior

arXiv: https://arxiv.org/abs/2609.03622 · HTML: https://mlsanity.com/paper/2609.03622/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 32 references that are in the map.
- First posted on arXiv 3 Sep 2026.

## Builds on (most cited first, 26 of 26)

- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [MEMO: Test Time Robustness via Adaptation and Augmentation](https://mlsanity.com/paper/2110.09506/) (2021)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors](https://mlsanity.com/paper/2110.01763/) (2021)
- [LibriMix: An Open-Source Dataset for Generalizable Speech Separation](https://mlsanity.com/paper/2005.11262/) (2020)
- [ICASSP 2023 Deep Noise Suppression Challenge](https://mlsanity.com/paper/2303.11510/) (2023)
- [EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2406.06185/) (2024)
- [Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration](https://mlsanity.com/paper/2107.05446/) (2021)
- [SELM: Speech Enhancement Using Discrete Tokens and Language Models](https://mlsanity.com/paper/2312.09747/) (2023)
- [A variance modeling framework based on variational autoencoders for speech enhancement](https://mlsanity.com/paper/1902.01605/) (2019)
- [Robustifying Vision Transformer without Retraining from Scratch by Test-Time Class-Conditional Feature Alignment](https://mlsanity.com/paper/2206.13951/) (2022)
- [RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing](https://mlsanity.com/paper/2202.08862/) (2022)
- [Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition](https://mlsanity.com/paper/2203.14222/) (2022)
- [Unsupervised Noise Adaptive Speech Enhancement by Discriminator-Constrained Optimal Transport](https://mlsanity.com/paper/2111.06316/) (2021)
- [Noise Adaptive Speech Enhancement using Domain Adversarial Training](https://mlsanity.com/paper/1807.07501/) (2018)
- [On Cross-Corpus Generalization of Deep Learning Based Speech Enhancement](https://mlsanity.com/paper/2002.04027/) (2020)
- [NU-GAN: High resolution neural upsampling with GAN](https://mlsanity.com/paper/2010.11362/) (2020)
- [Test-Time Adaptation Toward Personalized Speech Enhancement: Zero-Shot Learning with Knowledge Distillation](https://mlsanity.com/paper/2105.03544/) (2021)
- [Test-time Adaptation for Regression by Subspace Alignment](https://mlsanity.com/paper/2410.03263/) (2024)
- [Assessing the Generalization Gap of Learning-Based Speech Enhancement Systems in Noisy and Reverberant Environments](https://mlsanity.com/paper/2309.06183/) (2023)
- [Modeling strategies for speech enhancement in the latent space of a neural audio codec](https://mlsanity.com/paper/2510.26299/) (2025)
- [Test-Time Training for Speech Enhancement](https://mlsanity.com/paper/2508.01847/) (2025)
- [Test-Time Adaptation for Speech Enhancement via Domain Invariant Embedding Transformation](https://mlsanity.com/paper/2509.04280/) (2025)
- [Test-Time Adaptation For Speech Enhancement Via Mask Polarization](https://mlsanity.com/paper/2601.14770/) (2026)

## Abstract (arXiv)

Test-time adaptation (TTA) offers a promising direction for improving speech enhancement models under mismatched acoustic conditions, without requiring access to labeled target data. In this work, we propose a single-utterance TTA method that regularizes a pretrained speech enhancement model using an autoregressive prior trained on clean speech latent representations extracted from a neural audio codec. Adaptation is performed by minimizing the Kullback-Leibler divergence between the enhanced speech distribution and the clean speech prior. Experiments across multiple noisy speech datasets show consistent improvements in speech quality, particularly under training-testing noise mismatch conditions. Code and audio examples are available online.