# Thutmose Tagger: Single-pass neural model for Inverse Text Normalization

arXiv: https://arxiv.org/abs/2208.00064 · HTML: https://mlsanity.com/paper/2208.00064/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 29 Jul 2022.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Encode, Tag, Realize: High-Precision Text Editing](https://mlsanity.com/paper/1909.01187/) (2019)
- [RNN Approaches to Text Normalization: A Challenge](https://mlsanity.com/paper/1611.00068/) (2016)
- [Neural Inverse Text Normalization](https://mlsanity.com/paper/2102.06380/) (2021)
- [Improving Performance of End-to-End ASR on Numeric Sequences](https://mlsanity.com/paper/1907.01372/) (2019)
- [Proteno: Text Normalization with Limited Data for Fast Deployment in Text to Speech Systems](https://mlsanity.com/paper/2104.07777/) (2021)
- [A Unified Transformer-based Framework for Duplex Text Normalization](https://mlsanity.com/paper/2108.09889/) (2021)

## Built on by (most cited first, 5 of 5)

- [Dynamic Context-Aware Streaming Pretrained Language Model For Inverse Text Normalization](https://mlsanity.com/paper/2505.24229/) (2025)
- [Language Agnostic Data-Driven Inverse Text Normalization](https://mlsanity.com/paper/2301.08506/) (2023)
- [Improving Robustness of Neural Inverse Text Normalization via Data-Augmentation, Semi-Supervised Learning, and Post-Aligning Method](https://mlsanity.com/paper/2309.08626/) (2023)
- [Dual-Form ASR: Semantics-Aware Inverse Text Normalization for Chinese Speech Recognition](https://mlsanity.com/paper/2609.02901/) (2026)
- [Towards Efficient Simultaneous Inverse Text Normalization with Pretrained Text-to-Text Language Model and Read-Tag-Write Policy](https://mlsanity.com/paper/2610.19492/) (2026)

## Abstract (arXiv)

Inverse text normalization (ITN) is an essential post-processing step in automatic speech recognition (ASR). It converts numbers, dates, abbreviations, and other semiotic classes from the spoken form generated by ASR to their written forms. One can consider ITN as a Machine Translation task and use neural sequence-to-sequence models to solve it. Unfortunately, such neural models are prone to hallucinations that could lead to unacceptable errors. To mitigate this issue, we propose a single-pass token classifier model that regards ITN as a tagging task. The model assigns a replacement fragment to every input token or marks it for deletion or copying without changes. We present a dataset preparation method based on the granular alignment of ITN examples. The proposed model is less prone to hallucination errors. The model is trained on the Google Text Normalization dataset and achieves state-of-the-art sentence accuracy on both English and Russian test sets. One-to-one correspondence between tags and input words improves the interpretability of the model's predictions, simplifies debugging, and allows for post-processing corrections. The model is simpler than sequence-to-sequence models and easier to optimize in production settings. The model and the code to prepare the dataset is published as part of NeMo project.