# Positional Description for Numerical Normalization

arXiv: https://arxiv.org/abs/2408.12430 · HTML: https://mlsanity.com/paper/2408.12430/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 22 Aug 2024.

## Builds on (most cited first, 13 of 13)

- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Positional Description Matters for Transformers Arithmetic](https://mlsanity.com/paper/2311.14737/) (2023)
- [RNN Approaches to Text Normalization: A Challenge](https://mlsanity.com/paper/1611.00068/) (2016)
- [NeMo Inverse Text Normalization: From Development To Production](https://mlsanity.com/paper/2104.05055/) (2021)
- [Neural Inverse Text Normalization](https://mlsanity.com/paper/2102.06380/) (2021)
- [A Chat About Boring Problems: Studying GPT-based text normalization](https://mlsanity.com/paper/2309.13426/) (2023)
- [Scalable Multilingual Frontend for TTS](https://mlsanity.com/paper/2004.04934/) (2020)
- [Proteno: Text Normalization with Limited Data for Fast Deployment in Text to Speech Systems](https://mlsanity.com/paper/2104.07777/) (2021)
- [Text normalization using memory augmented neural networks](https://mlsanity.com/paper/1806.00044/) (2018)
- [A Unified Transformer-based Framework for Duplex Text Normalization](https://mlsanity.com/paper/2108.09889/) (2021)
- [Transformer-based Models of Text Normalization for Speech Applications](https://mlsanity.com/paper/2202.00153/) (2022)

## Abstract (arXiv)

We present a Positional Description Scheme (PDS) tailored for digit sequences, integrating placeholder value information for each digit. Given the structural limitations of subword tokenization algorithms, language models encounter critical Text Normalization (TN) challenges when handling numerical tasks. Our schema addresses this challenge through straightforward pre-processing, preserving the model architecture while significantly simplifying number normalization, rendering the problem tractable. This simplifies the task and facilitates more compact production-ready models capable of learning from smaller datasets. Furthermore, our investigations reveal that PDS enhances the arithmetic processing capabilities of language models, resulting in a relative accuracy improvement of 23% to 51% on complex arithmetic tasks. We demonstrate that PDS effectively mitigates fatal numerical normalization errors in neural models, requiring only a modest amount of training data without rule-based Finite State Transducers (FST). We demonstrate that PDS is essential for both the Text-To-Speech and Speech Recognition text processing, enabling effective TN under production constraints.