mlsanityData to 7 Oct 2026

Multilingual Contextual Adapters To Improve Custom Word Recognition In Low-resource Languages

arXiv 2307.00759 ↗First posted 3 Jul 2023cs.CL cs.SD eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

Connectionist Temporal Classification (CTC) models are popular for their balance between speed and performance for Automatic Speech Recognition (ASR). However, these CTC models still struggle in other areas, such as personalization towards custom words. A recent approach explores Contextual Adapters, wherein an attention-based biasing model for CTC is used to improve the recognition of custom entities. While this approach works well with enough data, we showcase that it isn't an effective strategy for low-resource languages. In this work, we propose a supervision loss for smoother training of the Contextual Adapters. Further, we explore a multilingual strategy to improve performance with limited training data. Our method achieves 48% F1 improvement in retrieving unseen custom entities for a low-resource language. Interestingly, as a by-product of training the Contextual Adapters, we see a 5-11% Word Error Rate (WER) reduction in the performance of the base CTC model as well.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 13 of its 19 references that are in the map. First posted on arXiv 3 Jul 2023.

Builds on

13 of 19 references.

References · 13

  1. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  2. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  3. 2018Deep context: end-to-end contextual speech recognitioncited by 136
  4. 2020Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameterscited by 110
  5. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 109
  6. 2017Multilingual Speech Recognition With A Single End-To-End Modelcited by 87
  7. 2020Deep Shallow Fusion for RNN-T Personalizationcited by 64
  8. 2022Contextual Adapters for Personalized Speech Recognition in Neural Transducerscited by 57
  9. 2018Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modelingcited by 34
  10. 2021Personalization Strategies for End-to-End Speech Recognition Systemscited by 26
  11. 2018Sequence-based Multi-lingual Low Resource Speech Recognitioncited by 20
  12. 2019Self-Attention Networks for Connectionist Temporal Classification in Speech Recognitioncited by 14
1 more
  1. 2017No Need for a Lexicon? Evaluating the Value of the Pronunciation Lexica in End-to-End Modelscited by 13

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Improving ASR Contextual Biasing with Guided Attentioncited by 17
  2. 2024Contextualized End-to-end Automatic Speech Recognition with Intermediate Biasing Losscited by 9
  3. 2025Improving Synthetic Data Training for Contextual Biasing Models with a Keyword-Aware Cost Functioncited by 1
  4. 2025A U-Net and Transformer Pipeline for Multilingual Image Translationcited by 0