# Contextualized Automatic Speech Recognition with Dynamic Vocabulary arXiv: https://arxiv.org/abs/2405.13344 ยท HTML: https://mlsanity.com/paper/2405.13344/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 33 of its 47 references that are in the map. - First posted on arXiv 22 May 2024. ## Builds on (most cited first, 30 of 33) - [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022) - [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020) - [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018) - [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015) - [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015) - [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021) - [Deep context: end-to-end contextual speech recognition](https://mlsanity.com/paper/1808.02480/) (2018) - [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023) - [An analysis of incorporating an external language model into a sequence-to-sequence model](https://mlsanity.com/paper/1712.01996/) (2017) - [Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion](https://mlsanity.com/paper/2104.02194/) (2021) - [Cold Fusion: Training Seq2Seq Models Together with Language Models](https://mlsanity.com/paper/1708.06426/) (2017) - [Contextual RNN-T For Open Domain ASR](https://mlsanity.com/paper/2006.03411/) (2020) - [OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer](https://mlsanity.com/paper/2401.16658/) (2024) - [Deliberation Model Based Two-Pass End-to-End Speech Recognition](https://mlsanity.com/paper/2003.07962/) (2020) - [Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LM](https://mlsanity.com/paper/1706.02737/) (2017) - [Class LM and word mapping for contextual biasing in End-to-End ASR](https://mlsanity.com/paper/2007.05609/) (2020) - [Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data](https://mlsanity.com/paper/2309.13876/) (2023) - [Improving End-to-End Contextual Speech Recognition with Fine-Grained Contextual Knowledge Selection](https://mlsanity.com/paper/2201.12806/) (2022) - [OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identification](https://mlsanity.com/paper/2402.12654/) (2024) - [Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognition](https://mlsanity.com/paper/2109.00627/) (2021) - [Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Network](https://mlsanity.com/paper/2305.12493/) (2023) - [Instant One-Shot Word-Learning for Context-Specific Neural Sequence-to-Sequence Speech Recognition](https://mlsanity.com/paper/2107.02268/) (2021) - [Transformer ASR with Contextual Block Processing](https://mlsanity.com/paper/1910.07204/) (2019) - [Contextualized Automatic Speech Recognition with Attention-Based Bias Phrase Boosted Beam Search](https://mlsanity.com/paper/2401.10449/) (2024) - [Towards Contextual Spelling Correction for Customization of End-to-end Speech Recognition Systems](https://mlsanity.com/paper/2203.00888/) (2022) - [Accelerating RNN-T Training and Inference Using CTC guidance](https://mlsanity.com/paper/2210.16481/) (2022) - [CopyNE: Better Contextual ASR by Copying Named Entities](https://mlsanity.com/paper/2305.12839/) (2023) - [Phoneme-aware Encoding for Prefix-tree-based Contextual ASR](https://mlsanity.com/paper/2312.09582/) (2023) - [Improving Contextual Spelling Correction by External Acoustics Attention and Semantic Aware Data Augmentation](https://mlsanity.com/paper/2302.11192/) (2023) - [4D ASR: Joint modeling of CTC, Attention, Transducer, and Mask-Predict decoders](https://mlsanity.com/paper/2212.10818/) (2022) ## Built on by (most cited first, 8 of 8) - [OWSM-Biasing: Contextualizing Open Whisper-Style Speech Models for Automatic Speech Recognition with Dynamic Vocabulary](https://mlsanity.com/paper/2506.09448/) (2025) - [DYNAC: Dynamic Vocabulary based Non-Autoregressive Contextualization for Speech Recognition](https://mlsanity.com/paper/2506.00422/) (2025) - [Contextualized Automatic Speech Recognition with Dynamic Vocabulary Prediction and Activation](https://mlsanity.com/paper/2505.23077/) (2025) - [Improving Contextual ASR via Multi-grained Fusion with Large Language Models](https://mlsanity.com/paper/2507.12252/) (2025) - [MGSC: A Multi-granularity Consistency Framework for Robust End-to-end Asr](https://mlsanity.com/paper/2508.15853/) (2025) - [Peeking Into The Future For Contextual Biasing](https://mlsanity.com/paper/2512.17657/) (2025) - [CALM: Joint Contextual Acoustic-Linguistic Modeling for Personalization of Multi-Speaker ASR](https://mlsanity.com/paper/2601.22792/) (2026) - [From Oracle to Noisy Context: Mitigating Contextual Exposure Bias in Speech-LLMs](https://mlsanity.com/paper/2603.24034/) (2026) ## Abstract (arXiv) Deep biasing (DB) enhances the performance of end-to-end automatic speech recognition (E2E-ASR) models for rare words or contextual phrases using a bias list. However, most existing methods treat bias phrases as sequences of subwords in a predefined static vocabulary. This naive sequence decomposition produces unnatural token patterns, significantly lowering their occurrence probability. More advanced techniques address this problem by expanding the vocabulary with additional modules, including the external language model shallow fusion or rescoring. However, they result in increasing the workload due to the additional modules. This paper proposes a dynamic vocabulary where bias tokens can be added during inference. Each entry in a bias list is represented as a single token, unlike a sequence of existing subword tokens. This approach eliminates the need to learn subword dependencies within the bias phrases. This method is easily applied to various architectures because it only expands the embedding and output layers in common E2E-ASR architectures. Experimental results demonstrate that the proposed method improves the bias phrase WER on English and Japanese datasets by 3.1 -- 4.9 points compared with the conventional DB method.