# Language Adaptive Cross-lingual Speech Representation Learning with Sparse Sharing Sub-networks

arXiv: https://arxiv.org/abs/2203.04583 · HTML: https://mlsanity.com/paper/2203.04583/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2022.

## Built on by (most cited first, 13 of 13)

- [LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASR](https://mlsanity.com/paper/2406.06619/) (2024)
- [Improving Massively Multilingual ASR With Auxiliary CTC Objectives](https://mlsanity.com/paper/2302.12829/) (2023)
- [Towards Zero-Shot Code-Switched Speech Recognition](https://mlsanity.com/paper/2211.01458/) (2022)
- [Examining Modularity in Multilingual LMs via Language-Specialized Subnetworks](https://mlsanity.com/paper/2311.08273/) (2023)
- [Learning ASR pathways: A sparse multilingual ASR model](https://mlsanity.com/paper/2209.05735/) (2022)
- [PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representations](https://mlsanity.com/paper/2203.16965/) (2022)
- [Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing](https://mlsanity.com/paper/2211.01522/) (2022)
- [Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation](https://mlsanity.com/paper/2502.17380/) (2025)
- [Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Model](https://mlsanity.com/paper/2309.13018/) (2023)
- [Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks](https://mlsanity.com/paper/2211.00106/) (2022)
- [SPEED: A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference](https://mlsanity.com/paper/2409.14017/) (2024)
- [Language-Aware Prompt Tuning for Parameter-Efficient Seamless Language Expansion in Multilingual ASR](https://mlsanity.com/paper/2506.21577/) (2025)
- [LaDA: Latent Dialogue Action For Zero-shot Cross-lingual Neural Network Language Modeling](https://mlsanity.com/paper/2308.02903/) (2023)

## Abstract (arXiv)

Unsupervised cross-lingual speech representation learning (XLSR) has recently shown promising results in speech recognition by leveraging vast amounts of unlabeled data across multiple languages. However, standard XLSR model suffers from language interference problem due to the lack of language specific modeling ability. In this work, we investigate language adaptive training on XLSR models. More importantly, we propose a novel language adaptive pre-training approach based on sparse sharing sub-networks. It makes room for language specific modeling by pruning out unimportant parameters for each language, without requiring any manually designed language specific component. After pruning, each language only maintains a sparse sub-network, while the sub-networks are partially shared with each other. Experimental results on a downstream multilingual speech recognition task show that our proposed method significantly outperforms baseline XLSR models on both high resource and low resource languages. Besides, our proposed method consistently outperforms other adaptation methods and requires fewer parameters.