mlsanityData to 14 Oct 2026

Test-Time Training for Speech

arXiv 2309.10930 ↗First posted 19 Sep 2023cs.SD cs.LG eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we study the application of Test-Time Training (TTT) as a solution to handling distribution shifts in speech applications. In particular, we introduce distribution-shifts to the test datasets of standard speech-classification tasks -- for example, speaker-identification and emotion-detection -- and explore how Test-Time Training (TTT) can help adjust to the distribution-shift. In our experiments that include distribution shifts due to background noise and natural variations in speech such as gender and age, we identify some key-challenges with TTT including sensitivity to optimization hyperparameters (e.g., number of optimization steps and subset of parameters chosen for TTT) and scalability (e.g., as each example gets its own set of parameters, TTT is not scalable). Finally, we propose using BitFit -- a parameter-efficient fine-tuning algorithm proposed for text applications that only considers the bias parameters for fine-tuning -- as a solution to the aforementioned challenges and demonstrate that it is consistently more stable than fine-tuning all the parameters of the model.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 50 of its 73 references that are in the map. First posted on arXiv 19 Sep 2023.

Builds on

50 of 73 references.

References · 50

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  8. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  9. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  10. 2022Visual Prompt Tuningcited by 1,681
  11. 2017Adversarial Discriminative Domain Adaptationcited by 1,311
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
38 more
  1. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  2. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  3. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  4. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  5. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  6. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  7. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  8. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  9. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 431
  10. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 407
  11. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 320
  12. 2022Masked Autoencoders that Listencited by 273
  13. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  14. 2020Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shiftcited by 236
  15. 2022Test-Time Training with Masked Autoencoderscited by 212
  16. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  17. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 198
  18. 2021SSAST: Self-Supervised Audio Spectrogram Transformercited by 169
  19. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 162
  20. 2022Test-Time Adaptation via Conjugate Pseudo-labelscited by 98
  21. 2022MAE-AST: Masked Autoencoding Audio Spectrogram Transformercited by 82
  22. 2020Rethinking Evaluation in ASR: Are Our Models Robust Enough?cited by 73
  23. 2021Test-Time Adaptation to Distribution Shift by Confidence Maximization and Input Transformationcited by 68
  24. 2021SITA: Single Image Test-time Adaptationcited by 59
  25. 2022Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representationcited by 52
  26. 2022Masked Spectrogram Prediction For Self-Supervised Audio Pre-Trainingcited by 51
  27. 2019A scalable noisy speech dataset and online subjective test frameworkcited by 48
  28. 2020Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Trainingcited by 39
  29. 2022Test-Time Training Can Close the Natural Distribution Shift Performance Gap in Deep Learning Based Compressed Sensingcited by 37
  30. 2022TTTFlow: Unsupervised Test-Time Training with Normalizing Flowcited by 33
  31. 2022Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Informationcited by 32
  32. 2021Self-Supervised Test-Time Learning for Reading Comprehensioncited by 22
  33. 2022SPIRAL: Self-supervised Perturbation-Invariant Representation Learning for Speech Pre-Trainingcited by 19
  34. 2022Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognitioncited by 19
  35. 2020REDAT: Accent-Invariant Representation for End-to-End ASR by Domain Adversarial Training with Relabelingcited by 18
  36. 2022Parameter-Efficient Tuning Makes a Good Classification Headcited by 15
  37. 2023In Search for a Generalizable Method for Source Free Domain Adaptationcited by 12
  38. 2022Resource-Efficient Transfer Learning From Speech Foundation Model Using Hierarchical Feature Fusioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Beyond Model Adaptation at Test Time: A Surveycited by 59
  2. 2026Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMscited by 11
  3. 2025Test-Time Training for Speech Enhancementcited by 2
  4. 2024Test-Time Training for Depression Detectioncited by 0