# Sparsely Shared LoRA on Whisper for Child Speech Recognition

arXiv: https://arxiv.org/abs/2309.11756 · HTML: https://mlsanity.com/paper/2309.11756/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2023.

## Built on by (most cited first, 18 of 18)

- [LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation](https://mlsanity.com/paper/2402.07721/) (2024)
- [Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models](https://mlsanity.com/paper/2406.10507/) (2024)
- [Federated LoRA with Sparse Communication](https://mlsanity.com/paper/2406.05233/) (2024)
- [Parameter Efficient Fine Tuning: A Comprehensive Analysis Across Applications](https://mlsanity.com/paper/2404.13506/) (2024)
- [Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on Whisper](https://mlsanity.com/paper/2408.10680/) (2024)
- [Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages](https://mlsanity.com/paper/2503.23542/) (2025)
- [Speech Recognition for Automatically Assessing Afrikaans and isiXhosa Preschool Oral Narratives](https://mlsanity.com/paper/2501.06478/) (2025)
- [SSVD: Structured SVD for Parameter-Efficient Fine-Tuning and Benchmarking under Domain Shift in ASR](https://mlsanity.com/paper/2509.02830/) (2025)
- [Adapter-Based Multi-Agent AVSR Extension for Pre-Trained ASR Models](https://mlsanity.com/paper/2502.01709/) (2025)
- [UME: Upcycling Mixture-of-Experts for Scalable and Efficient Automatic Speech Recognition](https://mlsanity.com/paper/2412.17507/) (2024)
- [Automated evaluation of children's speech fluency for low-resource languages](https://mlsanity.com/paper/2505.19671/) (2025)
- [Prompting Whisper for Improved Verbatim Transcription and End-to-end Miscue Detection](https://mlsanity.com/paper/2505.23627/) (2025)
- [A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASR](https://mlsanity.com/paper/2601.00557/) (2026)
- [Examining Test-Time Adaptation for Personalized Child Speech Recognition](https://mlsanity.com/paper/2409.13095/) (2024)
- [KSOD: Knowledge Supplement for LLMs On Demand](https://mlsanity.com/paper/2503.07550/) (2025)
- [Parameter-Efficient Fine-Tuning for HAR: Integrating LoRA and QLoRA into Transformer Models](https://mlsanity.com/paper/2512.17983/) (2025)
- [Robust Multi-Tier Infant-Centered Audio Understanding with Whisper via Structured Speaker Conditioning](https://mlsanity.com/paper/2608.11587/) (2026)
- [T-SANDHI: Tone Sandhi-aware Adaptive Network with Decoupled Hybrid Injection for Low-resource Taiwanese Hokkien Speech Recognition](https://mlsanity.com/paper/2609.18194/) (2026)

## Abstract (arXiv)

Whisper is a powerful automatic speech recognition (ASR) model. Nevertheless, its zero-shot performance on low-resource speech requires further improvement. Child speech, as a representative type of low-resource speech, is leveraged for adaptation. Recently, parameter-efficient fine-tuning (PEFT) in NLP was shown to be comparable and even better than full fine-tuning, while only needing to tune a small set of trainable parameters. However, current PEFT methods have not been well examined for their effectiveness on Whisper. In this paper, only parameter composition types of PEFT approaches such as LoRA and Bitfit are investigated as they do not bring extra inference costs. Different popular PEFT methods are examined. Particularly, we compare LoRA and AdaLoRA and figure out the learnable rank coefficient is a good design. Inspired by the sparse rank distribution allocated by AdaLoRA, a novel PEFT approach Sparsely Shared LoRA (S2-LoRA) is proposed. The two low-rank decomposed matrices are globally shared. Each weight matrix only has to maintain its specific rank coefficients that are constrained to be sparse. Experiments on low-resource Chinese child speech show that with much fewer trainable parameters, S2-LoRA can achieve comparable in-domain adaptation performance to AdaLoRA and exhibit better generalization ability on out-of-domain data. In addition, the rank distribution automatically learned by S2-LoRA is found to have similar patterns to AdaLoRA's allocation.