# LI-TTA: Language Informed Test-Time Adaptation for Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2408.05769 · HTML: https://mlsanity.com/paper/2408.05769/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 11 Aug 2024.

## Builds on (most cited first, 15 of 15)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models](https://mlsanity.com/paper/2209.07511/) (2022)
- [MEMO: Test Time Robustness via Adaptation and Augmentation](https://mlsanity.com/paper/2110.09506/) (2021)
- [TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation](https://mlsanity.com/paper/2302.05155/) (2023)
- [C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion](https://mlsanity.com/paper/2403.14119/) (2024)
- [A scalable noisy speech dataset and online subjective test framework](https://mlsanity.com/paper/1909.08050/) (2019)
- [Test time Adaptation through Perturbation Robustness](https://mlsanity.com/paper/2110.10232/) (2021)
- [Unsupervised Domain Adaptation for Robust Speech Recognition via Variational Autoencoder-Based Data Augmentation](https://mlsanity.com/paper/1707.06265/) (2017)
- [Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition](https://mlsanity.com/paper/2203.14222/) (2022)
- [Speech Technology for Everyone: Automatic Speech Recognition for Non-Native English with Transfer Learning](https://mlsanity.com/paper/2110.00678/) (2021)
- [Cross-domain Speech Recognition with Unsupervised Character-level Distribution Matching](https://mlsanity.com/paper/2104.07491/) (2021)
- [INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech Recognition](https://mlsanity.com/paper/2305.16371/) (2023)

## Built on by (most cited first, 3 of 3)

- [SUTA-LM: Bridging Test-Time Adaptation and Language Model Rescoring for Robust ASR](https://mlsanity.com/paper/2506.11121/) (2025)
- [StarDrinks: An English and Korean Test Set for SLU Evaluation in a Drink Ordering Scenario](https://mlsanity.com/paper/2604.26500/) (2026)
- [Rethinking Entropy Minimization in Test-Time Adaptation for Autoregressive Models](https://mlsanity.com/paper/2605.08186/) (2026)

## Abstract (arXiv)

Test-Time Adaptation (TTA) has emerged as a crucial solution to the domain shift challenge, wherein the target environment diverges from the original training environment. A prime exemplification is TTA for Automatic Speech Recognition (ASR), which enhances model performance by leveraging output prediction entropy minimization as a self-supervision signal. However, a key limitation of this self-supervision lies in its primary focus on acoustic features, with minimal attention to the linguistic properties of the input. To address this gap, we propose Language Informed Test-Time Adaptation (LI-TTA), which incorporates linguistic insights during TTA for ASR. LI-TTA integrates corrections from an external language model to merge linguistic with acoustic information by minimizing the CTC loss from the correction alongside the standard TTA loss. With extensive experiments, we show that LI-TTA effectively improves the performance of TTA for ASR in various distribution shift situations.