mlsanityData to 7 Oct 2026

Streaming, fast and accurate on-device Inverse Text Normalization for Automatic Speech Recognition

arXiv 2211.03721 ↗First posted 7 Nov 2022cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Automatic Speech Recognition (ASR) systems typically yield output in lexical form. However, humans prefer a written form output. To bridge this gap, ASR systems usually employ Inverse Text Normalization (ITN). In previous works, Weighted Finite State Transducers (WFST) have been employed to do ITN. WFSTs are nicely suited to this task but their size and run-time costs can make deployment on embedded applications challenging. In this paper, we describe the development of an on-device ITN system that is streaming, lightweight & accurate. At the core of our system is a streaming transformer tagger, that tags lexical tokens from ASR. The tag informs which ITN category might be applied, if at all. Following that, we apply an ITN-category-specific WFST, only on the tagged text, to reliably perform the ITN conversion. We show that the proposed ITN solution performs equivalent to strong baselines, while being significantly smaller in size and retaining customization capabilities.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 13 of its 26 references that are in the map. First posted on arXiv 7 Nov 2022.

Builds on

13 of 26 references.

References · 13

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  4. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
  5. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
  6. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 94
  7. 2016Training Sparse Neural Networkscited by 73
  8. 2020On the Comparison of Popular End-to-End Models for Large Scale Speech Recognitioncited by 62
  9. 2016Pruning Convolutional Neural Networks for Resource Efficient Inferencecited by 47
  10. 2017Exploring Neural Transducers for End-to-End Speech Recognitioncited by 43
  11. 2018Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterizationcited by 35
  12. 2016RNN Approaches to Text Normalization: A Challengecited by 30
1 more
  1. 2021Neural Inverse Text Normalizationcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 1; 2026: 1.

3 papers.

  1. 2025Dynamic Context-Aware Streaming Pretrained Language Model For Inverse Text Normalizationcited by 1
  2. 2023Language Agnostic Data-Driven Inverse Text Normalizationcited by 0
  3. 2026Dual-Form ASR: Semantics-Aware Inverse Text Normalization for Chinese Speech Recognitioncited by 0