mlsanityData to 7 Oct 2026

Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling

arXiv 1810.03459 ↗First posted 4 Oct 2018cs.CL cs.LG cs.SDcited by 34

Not on the map. Why?

Abstract (arXiv)

Sequence-to-sequence (seq2seq) approach for low-resource ASR is a relatively new direction in speech research. The approach benefits by performing model training without using lexicon and alignments. However, this poses a new problem of requiring more data compared to conventional DNN-HMM systems. In this work, we attempt to use data from 10 BABEL languages to build a multi-lingual seq2seq model as a prior model, and then port them towards 4 other BABEL languages using transfer learning approach. We also explore different architectures for improving the prior multilingual seq2seq model. The paper also discusses the effect of integrating a recurrent neural network language model (RNNLM) with a seq2seq model during decoding. Experimental results show that the transfer learning approach from the multilingual model shows substantial gains over monolingual models across all 4 BABEL languages. Incorporating an RNNLM also brings significant improvements in terms of %WER, and achieves recognition performance comparable to the models trained with twice more training data.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 15; 2022: 7; 2023: 6; 2024: 5; 2025: 1.

34 papers.

Sort:

  1. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  2. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  3. 2021PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognitioncited by 50
  4. 2024Automatic Speech Recognition using Advanced Deep Learning Approaches: A surveycited by 50
  5. 2021Towards One Model to Rule All: Multilingual Strategy for Dialectal Code-Switching Arabic ASRcited by 40
  6. 2023DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learningcited by 34
  7. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  8. 2021Exploiting Adapters for Cross-lingual Low-resource Speech Recognitioncited by 33
  9. 2023Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalizationcited by 28
  10. 2022Streaming End-to-End Multilingual Speech Recognition with Joint Language Identificationcited by 24
  11. 2021Pseudo-Labeling for Massively Multilingual Speech Recognitioncited by 23
  12. 2021Improved Language Identification Through Cross-Lingual Self-Supervised Learningcited by 20
22 more
  1. 2021Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognitioncited by 16
  2. 2022Maestro-U: Leveraging joint speech-text representation learning for zero supervised speech ASRcited by 13
  3. 2021XLST: Cross-lingual Self-training to Learn Multilingual Representation for Low Resource Speech Recognitioncited by 11
  4. 2022Combining Spectral and Self-Supervised Features for Low Resource Speech Recognition and Translationcited by 11
  5. 2021Cascaded Multilingual Audio-Visual Learning from Videoscited by 8
  6. 2024Wav2Gloss: Generating Interlinear Glossed Text from Speechcited by 8
  7. 2024Scaling A Simple Approach to Zero-Shot Speech Recognitioncited by 8
  8. 2021A Study of Multilingual End-to-End Speech Recognition for Kazakh, Russian, and Englishcited by 4
  9. 2023Multilingual Contextual Adapters To Improve Custom Word Recognition In Low-resource Languagescited by 4
  10. 2021Overcoming Domain Mismatch in Low Resource Sequence-to-Sequence ASR Models using Hybrid Generated Pseudotranscriptscited by 3
  11. 2022End-to-End Spoken Language Understanding: Performance analyses of a voice command task in a low resource settingcited by 3
  12. 2022Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challengescited by 3
  13. 2021Dynamic Acoustic Unit Augmentation With BPE-Dropout for Low-Resource End-to-End Speech Recognitioncited by 2
  14. 2022Non-Linear Pairwise Language Mappings for Low-Resource Multilingual Acoustic Model Fusioncited by 2
  15. 2023Cross-lingual Knowledge Transfer and Iterative Pseudo-labeling for Low-Resource Speech Recognition with Transducerscited by 2
  16. 2023A Quantitative Approach to Understand Self-Supervised Models as Cross-lingual Feature Extractorscited by 2
  17. 2021Two-Stage Augmentation and Adaptive CTC Fusion for Improved Robustness of Multi-Stream End-to-End ASRcited by 1
  18. 2021Train your classifier first: Cascade Neural Networks Training from upper layers to lower layerscited by 1
  19. 2022Curriculum optimization for low-resource speech recognitioncited by 1
  20. 2024Speech Prefix-Tuning with RNNT Loss for Improving LLM Predictionscited by 1
  21. 2025Flavors of Moonshine: Tiny Specialized ASR Models for Edge Devicescited by 1
  22. 2021Coarse-To-Fine And Cross-Lingual ASR Transfercited by 0