# Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Network

arXiv: https://arxiv.org/abs/2305.12493 · HTML: https://mlsanity.com/paper/2305.12493/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 21 May 2023.

## Builds on (most cited first, 18 of 18)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio](https://mlsanity.com/paper/2106.06909/) (2021)
- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [Deep context: end-to-end contextual speech recognition](https://mlsanity.com/paper/1808.02480/) (2018)
- [Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion](https://mlsanity.com/paper/2104.02194/) (2021)
- [Context-Aware Transformer Transducer for Speech Recognition](https://mlsanity.com/paper/2111.03250/) (2021)
- [WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2203.15455/) (2022)
- [Class LM and word mapping for contextual biasing in End-to-End ASR](https://mlsanity.com/paper/2007.05609/) (2020)
- [Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognition](https://mlsanity.com/paper/2109.00627/) (2021)
- [End-to-end contextual speech recognition using class language models and a token passing decoder](https://mlsanity.com/paper/1812.02142/) (2018)
- [Improved Neural Language Model Fusion for Streaming Recurrent Neural Network Transducer](https://mlsanity.com/paper/2010.13878/) (2020)
- [CIF-based Collaborative Decoding for End-to-end Contextual Speech Recognition](https://mlsanity.com/paper/2012.09466/) (2020)
- [Tiny Transducer: A Highly-efficient Speech Recognition Model on Edge Devices](https://mlsanity.com/paper/2101.06856/) (2021)
- [Improving Contextual Recognition of Rare Words with an Alternate Spelling Prediction Model](https://mlsanity.com/paper/2209.01250/) (2022)
- [Two Stage Contextual Word Filtering for Context bias in Unified Streaming and Non-streaming Transducer](https://mlsanity.com/paper/2301.06735/) (2023)

## Built on by (most cited first, 30 of 32)

- [PromptASR for contextualized ASR with controllable style](https://mlsanity.com/paper/2309.07414/) (2023)
- [Contextualized Automatic Speech Recognition with Attention-Based Bias Phrase Boosted Beam Search](https://mlsanity.com/paper/2401.10449/) (2024)
- [CTC-Assisted LLM-Based Contextual ASR](https://mlsanity.com/paper/2411.06437/) (2024)
- [SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Ability](https://mlsanity.com/paper/2308.03266/) (2023)
- [Improving ASR Contextual Biasing with Guided Attention](https://mlsanity.com/paper/2401.08835/) (2024)
- [Contextualized End-to-end Automatic Speech Recognition with Intermediate Biasing Loss](https://mlsanity.com/paper/2406.16120/) (2024)
- [Contextualized Automatic Speech Recognition with Dynamic Vocabulary](https://mlsanity.com/paper/2405.13344/) (2024)
- [SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processing](https://mlsanity.com/paper/2601.09385/) (2026)
- [Low-Resource Domain Adaptation for Speech LLMs via Text-Only Fine-Tuning](https://mlsanity.com/paper/2506.05671/) (2025)
- [Improving Large-scale Deep Biasing with Phoneme Features and Text-only Data in Streaming Transducer](https://mlsanity.com/paper/2311.08966/) (2023)
- [Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions](https://mlsanity.com/paper/2407.17874/) (2024)
- [InterBiasing: Boost Unseen Word Recognition through Biasing Intermediate Predictions](https://mlsanity.com/paper/2406.14890/) (2024)
- [OWSM-Biasing: Contextualizing Open Whisper-Style Speech Models for Automatic Speech Recognition with Dynamic Vocabulary](https://mlsanity.com/paper/2506.09448/) (2025)
- [Optimizing Contextual Speech Recognition Using Vector Quantization for Efficient Retrieval](https://mlsanity.com/paper/2411.00664/) (2024)
- [WCTC-Biasing: Retraining-free Contextual Biasing ASR with Wildcard CTC-based Keyword Spotting and Inter-layer Biasing](https://mlsanity.com/paper/2506.01263/) (2025)
- [Zero-shot Context Biasing with Trie-based Decoding using Synthetic Multi-Pronunciation](https://mlsanity.com/paper/2508.17796/) (2025)
- [DYNAC: Dynamic Vocabulary based Non-Autoregressive Contextualization for Speech Recognition](https://mlsanity.com/paper/2506.00422/) (2025)
- [Automatic Speech Recognition of African American English: Lexical and Contextual Effects](https://mlsanity.com/paper/2506.06888/) (2025)
- [Improving Contextual ASR via Multi-grained Fusion with Large Language Models](https://mlsanity.com/paper/2507.12252/) (2025)
- [Contextualized Automatic Speech Recognition with Dynamic Vocabulary Prediction and Activation](https://mlsanity.com/paper/2505.23077/) (2025)
- [CMT-LLM: Contextual Multi-Talker ASR Utilizing Large Language Models](https://mlsanity.com/paper/2506.12059/) (2025)
- [Enhancing the Robustness of Contextual ASR to Varying Biasing Information Volumes Through Purified Semantic Correlation Joint Modeling](https://mlsanity.com/paper/2509.05908/) (2025)
- [Dolphin-CN-Dialect: Where Chinese Dialects Matter](https://mlsanity.com/paper/2605.08961/) (2026)
- [How to Recognize New Words: A Comparison Between Context Biasing Methods and Speech LLMs](https://mlsanity.com/paper/2608.05759/) (2026)
- [Automatic Text Pronunciation Correlation Generation and Application for Contextual Biasing](https://mlsanity.com/paper/2501.00804/) (2025)
- [Context Biasing for Pronunciation-Orthography Mismatch in Automatic Speech Recognition](https://mlsanity.com/paper/2506.18703/) (2025)
- [Speech-Aware Long Context Pruning and Integration for Contextualized Automatic Speech Recognition](https://mlsanity.com/paper/2511.11139/) (2025)
- [Peeking Into The Future For Contextual Biasing](https://mlsanity.com/paper/2512.17657/) (2025)
- [CALM: Joint Contextual Acoustic-Linguistic Modeling for Personalization of Multi-Speaker ASR](https://mlsanity.com/paper/2601.22792/) (2026)
- [Speak in Context: Multilingual ASR with Speech Context Alignment via Contrastive Learning](https://mlsanity.com/paper/2603.06505/) (2026)

## Abstract (arXiv)

Contextual information plays a crucial role in speech recognition technologies and incorporating it into the end-to-end speech recognition models has drawn immense interest recently. However, previous deep bias methods lacked explicit supervision for bias tasks. In this study, we introduce a contextual phrase prediction network for an attention-based deep bias method. This network predicts context phrases in utterances using contextual embeddings and calculates bias loss to assist in the training of the contextualized model. Our method achieved a significant word error rate (WER) reduction across various end-to-end speech recognition models. Experiments on the LibriSpeech corpus show that our proposed model obtains a 12.1% relative WER improvement over the baseline model, and the WER of the context phrases decreases relatively by 40.5%. Moreover, by applying a context phrase filtering strategy, we also effectively eliminate the WER degradation when using a larger biasing list.