mlsanityData to 14 Oct 2026

Streaming End-to-End Multilingual Speech Recognition with Joint Language Identification

arXiv 2209.06058 ↗First posted 13 Sep 2022eess.AS cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Language identification is critical for many downstream tasks in automatic speech recognition (ASR), and is beneficial to integrate into multilingual end-to-end ASR as an additional task. In this paper, we propose to modify the structure of the cascaded-encoder-based recurrent neural network transducer (RNN-T) model by integrating a per-frame language identifier (LID) predictor. RNN-T with cascaded encoders can achieve streaming ASR with low latency using first-pass decoding with no right-context, and achieve lower word error rates (WERs) using second-pass decoding with longer right-context. By leveraging such differences in the right-contexts and a streaming implementation of statistics pooling, the proposed method can achieve accurate streaming LID prediction with little extra test-time cost. Experimental results on a voice search dataset with 9 language locales shows that the proposed method achieves an average of 96.2% LID prediction accuracy and the same second-pass WER as that obtained by including oracle LID in the input.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 13 Sep 2022.

Builds on

23 of 41 references.

References · 23

  1. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  2. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  3. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 234
  4. 2020Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameterscited by 110
  5. 2017Multilingual Speech Recognition With A Single End-To-End Modelcited by 87
  6. 2021Scaling End-to-End Models for Large-Scale Multilingual ASRcited by 61
  7. 2020Cascaded encoders for unifying streaming and non-streaming ASRcited by 53
  8. 2018Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modelingcited by 34
  9. 2018On the End-to-End Solution to Mandarin-English Code-switching Speech Recognitioncited by 30
  10. 2021A Configurable Multilingual Model is All You Need to Recognize All Languagescited by 28
  11. 2020Transformer-Transducers for Code-Switched Speech Recognitioncited by 27
  12. 2020Streaming End-to-End Bilingual ASR Systems with Joint Language Identificationcited by 20
11 more
  1. 2021Improved Language Identification Through Cross-Lingual Self-Supervised Learningcited by 20
  2. 2021Joint Modeling of Code-Switched and Monolingual ASR via Conditional Factorizationcited by 18
  3. 2018Towards End-to-End Code-Switching Speech Recognitioncited by 17
  4. 2017Towards Language-Universal End-to-End Speech Recognitioncited by 14
  5. 2018Joint Modeling of Accents and Acoustics for Multi-Accent Speech Recognitioncited by 14
  6. 2020Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognitioncited by 11
  7. 2019Signal Combination for Language Identificationcited by 4
  8. 2020Streaming Language Identification using Combination of Acoustic Representations and ASR Hypothesescited by 4
  9. 2018Tuplemax Loss for Language Identificationcited by 3
  10. 2019Adversarial Training for Multilingual Acoustic Modelingcited by 3
  11. 2020Improving Language Identification for Multilingual Speakerscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 12; 2024: 7; 2025: 2; 2026: 2.

24 papers.

Sort:

By area:

  1. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 71
  2. 2023Improving Massively Multilingual ASR With Auxiliary CTC Objectivescited by 31
  3. 2023Mixture-of-Expert Conformer for Streaming Multilingual ASRcited by 17
  4. 2022Towards Zero-Shot Code-Switched Speech Recognitioncited by 15
  5. 2025Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translationcited by 6
  6. 2023Building High-accuracy Multilingual ASR with Gated Language Experts and Curriculum Trainingcited by 4
  7. 2023SSHR: Leveraging Self-supervised Hierarchical Representations for Multilingual Automatic Speech Recognitioncited by 4
  8. 2023UML: A Universal Monolingual Output Layer for Multilingual ASRcited by 3
  9. 2023Simple yet Effective Code-Switching Language Identification with Multitask Pre-Training and Transfer Learningcited by 2
  10. 2023Improving Code-Switching and Named Entity Recognition in ASR with Speech Editing based Data Augmentationcited by 2
  11. 2023Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identificationcited by 2
  12. 2024Improving Zero-Shot Chinese-English Code-Switching ASR with kNN-CTC and Gated Monolingual Datastorescited by 2
12 more
  1. 2023Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adaptercited by 0
  2. 2023GMTR: Graph Matching Transformerscited by 0
  3. 2023DarkShot: Lighting Dark Images with Low-Compute and High-Qualitycited by 0
  4. 2024TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processingcited by 0
  5. 2024TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networkscited by 0
  6. 2024T-Pixel2Mesh: Combining Global and Local Transformer for 3D Mesh Generation from a Single Imagecited by 0
  7. 2024Strategic Arms with Side Communication Prevail Over Low-Regret MAB Algorithmscited by 0
  8. 2024Towards General Deepfake Detection with Dynamic Curriculumcited by 0
  9. 2024Attr-Int: A Simple and Effective Entity Alignment Framework for Heterogeneous Knowledge Graphscited by 0
  10. 2025Mask6D: Masked Pose Priors For 6D Object Pose Estimationcited by 0
  11. 2026Are you speaking my languages? On spoken language adherence in multimodal LLMscited by 0
  12. 2026Jointly Improving Dialect Identification and ASR in Indian Languages using Multimodal Feature Fusioncited by 0