# Continual Adaptation for Pacific Indigenous Speech Recognition

arXiv: https://arxiv.org/abs/2603.06310 · HTML: https://mlsanity.com/paper/2603.06310/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 6 Mar 2026.

## Builds on (most cited first, 24 of 24)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [DoRA: Weight-Decomposed Low-Rank Adaptation](https://mlsanity.com/paper/2402.09353/) (2024)
- [Orthogonal Subspace Learning for Language Model Continual Learning](https://mlsanity.com/paper/2310.14152/) (2023)
- [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021)
- [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022)
- [Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages](https://mlsanity.com/paper/2511.09690/) (2025)
- [ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets](https://mlsanity.com/paper/2406.08641/) (2024)
- [A Survey of Multilingual Models for Automatic Speech Recognition](https://mlsanity.com/paper/2202.12576/) (2022)
- [CL-MASR: A Continual Learning Benchmark for Multilingual ASR](https://mlsanity.com/paper/2310.16931/) (2023)
- [Cross-lingual Transfer for Speech Processing using Acoustic Language Similarity](https://mlsanity.com/paper/2111.01326/) (2021)
- [Rainbow Keywords: Efficient Incremental Learning for Online Spoken Keyword Spotting](https://mlsanity.com/paper/2203.16361/) (2022)
- [CLEP-DG: Contrastive Learning for Speech Emotion Domain Generalization via Soft Prompt Tuning](https://mlsanity.com/paper/2507.04048/) (2025)
- [Continual Learning For On-Device Environmental Sound Classification](https://mlsanity.com/paper/2207.07429/) (2022)
- [Dark Experience for Incremental Keyword Spotting](https://mlsanity.com/paper/2409.08153/) (2024)
- [AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spotting](https://mlsanity.com/paper/2505.11817/) (2025)
- [Characterization of Speech Similarity Between Australian Aboriginal and High-Resource Languages: A Case Study on Dharawal](https://mlsanity.com/paper/2509.01419/) (2025)
- [DG-SED: Domain Generalization for Sound Event Detection with Heterogeneous Training Data](https://mlsanity.com/paper/2407.03654/) (2024)
- [AdaKWS: Towards Robust Keyword Spotting with Test-Time Adaptation](https://mlsanity.com/paper/2505.14600/) (2025)
- [AFT: An Exemplar-Free Class Incremental Learning Method for Environmental Sound Classification](https://mlsanity.com/paper/2509.15523/) (2025)
- [Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languages](https://mlsanity.com/paper/2602.01008/) (2026)

## Abstract (arXiv)

Speech foundation models struggle with low-resource Pacific Indigenous languages because of severe data scarcity. Furthermore, full fine-tuning risks catastrophic forgetting. To address this gap, we present an empirical study adapting models to real-world Pacific datasets. We investigate the impact of data volume, adaptation strategies, and representational drift on speech foundation models for various Pacific languages. Additionally, we analyze a continual learning framework for sequential language acquisition. Empirical results across three distinct Pacific Indigenous languages demonstrate that adapting to these linguistically distant languages induces severe internal representational drift. Consequently, these models face a strict plasticity and stability dilemma. While LoRA adapts well initially, it suffers from catastrophic forgetting during sequential learning. Ultimately, this study highlights the urgent need for robust adaptation strategies tailored to underrepresented languages.