# E-BATS: Efficient Backpropagation-Free Test-Time Adaptation for Speech Foundation Models

arXiv: https://arxiv.org/abs/2506.07078 · HTML: https://mlsanity.com/paper/2506.07078/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 8 Jun 2025.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [SWAD: Domain Generalization by Seeking Flat Minima](https://mlsanity.com/paper/2102.08604/) (2021)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation](https://mlsanity.com/paper/1805.04699/) (2018)
- [Parameter-free Online Test-time Adaptation](https://mlsanity.com/paper/2201.05718/) (2022)
- [The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization](https://mlsanity.com/paper/2112.00463/) (2021)
- [Test-Time Model Adaptation with Only Forward Passes](https://mlsanity.com/paper/2404.01650/) (2024)
- [DELTA: degradation-free fully test-time adaptation](https://mlsanity.com/paper/2301.13018/) (2023)
- [Subspace Identification for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2310.04723/) (2023)
- [Domain Generalization by Learning and Removing Domain-specific Features](https://mlsanity.com/paper/2212.07101/) (2022)
- [Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition](https://mlsanity.com/paper/2203.14222/) (2022)
- [HyperDomainNet: Universal Domain Adaptation for Generative Adversarial Networks](https://mlsanity.com/paper/2210.08884/) (2022)
- [Continual Test-time Adaptation for End-to-end Speech Recognition on Noisy Speech](https://mlsanity.com/paper/2406.11064/) (2024)
- [Unraveling Batch Normalization for Realistic Test-Time Adaptation](https://mlsanity.com/paper/2312.09486/) (2023)
- [Advancing Test-Time Adaptation in Wild Acoustic Test Settings](https://mlsanity.com/paper/2310.09505/) (2023)

## Built on by (most cited first, 8 of 8)

- [NEO: No-Optimization Test-Time Adaptation through Latent Re-Centering](https://mlsanity.com/paper/2510.05635/) (2025)
- [AdaNODEs: Test Time Adaptation for Time Series Forecasting Using Neural ODEs](https://mlsanity.com/paper/2601.12893/) (2026)
- [AudioMosaic: Contrastive Masked Audio Representation Learning](https://mlsanity.com/paper/2605.14231/) (2026)
- [Test-Time Adaptation for Speech Emotion Recognition](https://mlsanity.com/paper/2601.16240/) (2026)
- [Test-Time Adaptation for EEG Foundation Models: A Systematic Study under Real-World Distribution Shifts](https://mlsanity.com/paper/2604.16926/) (2026)
- [Prototype-Rectified Iterative Self-supervised Manifold Denoising under Severe Acoustic Shift](https://mlsanity.com/paper/2608.15037/) (2026)
- [RINSE: Robust Target-Time Normality Estimation for Zero-Shot Graph Anomaly Detection](https://mlsanity.com/paper/2609.02497/) (2026)
- [On the Interaction Between Model Compression and Test-Time Adaptation](https://mlsanity.com/paper/2609.03604/) (2026)

## Abstract (arXiv)

Speech Foundation Models encounter significant performance degradation when deployed in real-world scenarios involving acoustic domain shifts, such as background noise and speaker accents. Test-time adaptation (TTA) has recently emerged as a viable strategy to address such domain shifts at inference time without requiring access to source data or labels. However, existing TTA approaches, particularly those relying on backpropagation, are memory-intensive, limiting their applicability in speech tasks and resource-constrained settings. Although backpropagation-free methods offer improved efficiency, existing ones exhibit poor accuracy. This is because they are predominantly developed for vision tasks, which fundamentally differ from speech task formulations, noise characteristics, and model architecture, posing unique transferability challenges. In this paper, we introduce E-BATS, the first Efficient BAckpropagation-free TTA framework designed explicitly for speech foundation models. E-BATS achieves a balance between adaptation effectiveness and memory efficiency through three key components: (i) lightweight prompt adaptation for a forward-pass-based feature alignment, (ii) a multi-scale loss to capture both global (utterance-level) and local distribution shifts (token-level) and (iii) a test-time exponential moving average mechanism for stable adaptation across utterances. Experiments conducted on four noisy speech datasets spanning sixteen acoustic conditions demonstrate consistent improvements, with 4.1%-13.5% accuracy gains over backpropagation-free baselines and 2.0-6.4 times GPU memory savings compared to backpropagation-based methods. By enabling scalable and robust adaptation under acoustic variability, this work paves the way for developing more efficient adaptation approaches for practical speech processing systems in real-world environments.