Language-Universal Adapter Learning with Knowledge Distillation for End-to-End Multilingual Speech Recognition
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose a language-universal adapter learning framework based on a pre-trained model for end-to-end multilingual automatic speech recognition (ASR). For acoustic modeling, the wav2vec 2.0 pre-trained model is fine-tuned by inserting language-specific and language-universal adapters. An online knowledge distillation is then used to enable the language-universal adapters to learn both language-specific and universal features. The linguistic information confusion is also reduced by leveraging language identifiers (LIDs). With LIDs we perform a position-wise modification on the multi-head attention outputs. In the inference procedure, the language-specific adapters are removed while the language-universal adapters are kept activated. The proposed method improves the recognition accuracy and addresses the linear increase of the number of adapters' parameters with the number of languages in common multilingual ASR systems. Experiments on the BABEL dataset confirm the effectiveness of the proposed framework. Compared to the conventional multilingual model, a 3.3% absolute error rate reduction is achieved. The code is available at: https://github.com/shen9712/UniversalAdapterLearning.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 12 of its 19 references that are in the map. First posted on arXiv 28 Feb 2023.
Builds on
12 of 19 references.
References · 12
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
- 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
- 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
- 2019Self-Training for End-to-End Speech Recognitioncited by 128
- 2020UNKs Everywhere: Adapting Multilingual Language Models to New Scriptscited by 113
- 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 92
- 2019CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognitioncited by 76
- 2020Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognitioncited by 34
- 2020Transformer-based Online CTC/attention End-to-End Speech Recognition Architecturecited by 28
- 2021A Configurable Multilingual Model is All You Need to Recognize All Languagescited by 28
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.