mlsanityData to 14 Oct 2026

How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario

arXiv 2411.18217 ↗First posted 27 Nov 2024cs.SD cs.CL eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

The utilization of speech Self-Supervised Learning (SSL) models achieves impressive performance on Automatic Speech Recognition (ASR). However, in low-resource language ASR, they encounter the domain mismatch problem between pre-trained and low-resource languages. Typical solutions like fine-tuning the SSL model suffer from high computation costs while using frozen SSL models as feature extractors comes with poor performance. To handle these issues, we extend a conventional efficient fine-tuning scheme based on the adapter. We add an extra intermediate adaptation to warm up the adapter and downstream model initialization. Remarkably, we update only 1-5% of the total model parameters to achieve the adaptation. Experimental results on the ML-SUPERB dataset show that our solution outperforms conventional efficient fine-tuning. It achieves up to a 28% relative improvement in the Character/Phoneme error rate when adapting to unseen languages.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

27 of 32 references.

References · 27

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  8. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  9. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  10. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  11. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
  12. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
15 more
  1. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  2. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 71
  3. 2021Exploiting Adapters for Cross-lingual Low-resource Speech Recognitioncited by 35
  4. 2022Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognitioncited by 32
  5. 2022AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Taskscited by 29
  6. 2019Massively Multilingual Adversarial Speech Recognitioncited by 28
  7. 2022Exploring Efficient-tuning Methods in Self-supervised Speech Modelscited by 26
  8. 2022A Survey of Multilingual Models for Automatic Speech Recognitioncited by 21
  9. 2021An Adapter Based Pre-Training for Efficient and Scalable Self-Supervised Speech Representation Learningcited by 19
  10. 2023Multi-resolution HuBERT: Multi-resolution Speech Self-Supervised Learning with Masked Unit Predictioncited by 17
  11. 2022CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Modelscited by 13
  12. 2023Efficient Domain Adaptation for Speech Foundation Modelscited by 13
  13. 2021Cross-lingual Transfer for Speech Processing using Acoustic Language Similaritycited by 8
  14. 2023A Novel Self-training Approach for Low-resource Speech Recognitioncited by 7
  15. 2022Analyzing the factors affecting usefulness of Self-Supervised Pre-trained Representations for Speech Recognitioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Fake-Mamba: Real-Time Speech Deepfake Detection Using Bidirectional Mamba as Self-Attention's Alternativecited by 6
  2. 2026Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Taskscited by 1
  3. 2026SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Modelscited by 1
  4. 2025Exploring Dynamic Parameters for Vietnamese Gender-Independent ASRcited by 0
  5. 2026Dissecting Sensitivity to Training Language in Self-Supervised Speech Learning Using Neural Audio Codec Tokenscited by 0