# Rehearsal-Free Online Continual Learning for Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2306.10860 · HTML: https://mlsanity.com/paper/2306.10860/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2023.

## Built on by (most cited first, 7 of 7)

- [Continual Learning Optimizations for Auto-regressive Decoder of Multilingual ASR systems](https://mlsanity.com/paper/2407.03645/) (2024)
- [Weight Factorization and Centralization for Continual Learning in Speech Recognition](https://mlsanity.com/paper/2506.16574/) (2025)
- [Language-Aware Prompt Tuning for Parameter-Efficient Seamless Language Expansion in Multilingual ASR](https://mlsanity.com/paper/2506.21577/) (2025)
- [Sequential Editing for Lifelong Training of Speech Recognition Models](https://mlsanity.com/paper/2406.17935/) (2024)
- [Learning to Hear Hesitation: Continual Learning for Disfluency-Aware ASR](https://mlsanity.com/paper/2606.14391/) (2026)
- [MoDiCoL: A Modular Diagnostic Continual Learning Dataset for Robust Speech Recognition](https://mlsanity.com/paper/2606.14459/) (2026)
- [Speech-to-SOAP: End-to-End Summarization of Medical Dialogues: KIT@BeTraC 2026](https://mlsanity.com/paper/2608.24327/) (2026)

## Abstract (arXiv)

Fine-tuning an Automatic Speech Recognition (ASR) model to new domains results in degradation on original domains, referred to as Catastrophic Forgetting (CF). Continual Learning (CL) attempts to train ASR models without suffering from CF. While in ASR, offline CL is usually considered, online CL is a more realistic but also more challenging scenario where the model, unlike in offline CL, does not know when a task boundary occurs. Rehearsal-based methods, which store previously seen utterances in a memory, are often considered for online CL, in ASR and other research domains. However, recent research has shown that weight averaging is an effective method for offline CL in ASR. Based on this result, we propose, in this paper, a rehearsal-free method applicable for online CL. Our method outperforms all baselines, including rehearsal-based methods, in two experiments. Our method is a next step towards general CL for ASR, which should enable CL in all scenarios with few if any constraints.