mlsanityData to 7 Oct 2026

A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systems

arXiv 2108.07493 ↗First posted 17 Aug 2021cs.CL cs.LG cs.SDcited by 11

Not on the map. Why?

Abstract (arXiv)

It's challenging to customize transducer-based automatic speech recognition (ASR) system with context information which is dynamic and unavailable during model training. In this work, we introduce a light-weight contextual spelling correction model to correct context-related recognition errors in transducer-based ASR systems. We incorporate the context information into the spelling correction model with a shared context encoder and use a filtering algorithm to handle large-size context lists. Experiments show that the model improves baseline ASR model performance with about 50% relative word error rate reduction, which also significantly outperforms the baseline method such as contextual LM biasing. The model also shows excellent performance for out-of-vocabulary terms not seen during training.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 22 of its 37 references that are in the map. First posted on arXiv 17 Aug 2021.

Builds on

22 of 37 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2016Layer Normalizationcited by 5,579
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2015Listen, Attend and Spellcited by 542
  8. 2018A Survey on Deep Learning for Named Entity Recognitioncited by 281
  9. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
  10. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  11. 2018Deep context: end-to-end contextual speech recognitioncited by 136
  12. 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
10 more
  1. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 94
  2. 2020Contextual RNN-T For Open Domain ASRcited by 85
  3. 2019Improving RNN Transducer Modeling for End-to-End Speech Recognitioncited by 68
  4. 2019Transformer-Transducer: End-to-End Speech Recognition with Self-Attentioncited by 68
  5. 2020Deep Shallow Fusion for RNN-T Personalizationcited by 64
  6. 2019A spelling correction model for end-to-end speech recognitioncited by 63
  7. 2020Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capabilitycited by 63
  8. 2020A New Training Pipeline for an Improved Neural Transducercited by 34
  9. 2017Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Modelscited by 19
  10. 2019G2G: TTS-Driven Pronunciation Learning for Graphemic Hybrid ASRcited by 11

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2; 2023: 3; 2024: 3; 2025: 1.

11 papers.

  1. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  2. 2022Towards Contextual Spelling Correction for Customization of End-to-end Speech Recognition Systemscited by 17
  3. 2023Improving Contextual Spelling Correction by External Acoustics Attention and Semantic Aware Data Augmentationcited by 9
  4. 2021Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognitioncited by 7
  5. 2024Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASRcited by 5
  6. 2023SpellMapper: A non-autoregressive neural spellchecker for ASR customization with candidate retrieval based on n-gram mappingscited by 4
  7. 2022Can Visual Context Improve Automatic Speech Recognition for an Embodied Agent?cited by 3
  8. 2024Locality enhanced dynamic biasing and sampling strategies for contextual ASRcited by 3
  9. 2023Beyond Hard Samples: Robust and Effective Grammatical Error Correction with Cycle Self-Augmentingcited by 2
  10. 2024Spelling Correction through Rewriting of Non-Autoregressive ASR Latticescited by 1
  11. 2025PMF-CEC: Phoneme-augmented Multimodal Fusion for Context-aware ASR Error Correction with Error-specific Selective Decodingcited by 1