# A cost minimization approach to fix the vocabulary size in a tokenizer for an End-to-End ASR system

arXiv: https://arxiv.org/abs/2406.02563 · HTML: https://mlsanity.com/paper/2406.02563/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 14 references that are in the map.
- First posted on arXiv 29 Apr 2024.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [A Formal Perspective on Byte-Pair Encoding](https://mlsanity.com/paper/2306.16837/) (2023)
- [Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling](https://mlsanity.com/paper/1811.04284/) (2018)
- [Analyzing and Mitigating Bias for Vulnerable Classes: Towards Balanced Representation in Dataset](https://mlsanity.com/paper/2401.10397/) (2024)
- [Towards Consistent Hybrid HMM Acoustic Modeling](https://mlsanity.com/paper/2104.02387/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Calculus-Based Framework for Determining Vocabulary Size in End-to-End ASR](https://mlsanity.com/paper/2605.14427/) (2026)

## Abstract (arXiv)

Unlike hybrid speech recognition systems where the use of tokens was restricted to phones, biphones or triphones the choice of tokens in the end-to-end ASR systems is derived from the text corpus of the training data. The use of tokenization algorithms like Byte Pair Encoding (BPE) and WordPiece is popular in identifying the tokens that are used in the overall training process of the speech recognition system. Popular toolkits, like ESPNet use a pre-defined vocabulary size (number of tokens) for these tokenization algorithms, but there is no discussion on how vocabulary size was derived. In this paper, we build a cost function, assuming the tokenization process to be a black-box to enable choosing the number of tokens which might most benefit building an end-to-end ASR. We show through experiments on LibriSpeech 100 hour set that the performance of an end-to-end ASR system improves when the number of tokens are chosen carefully.